-- Logs begin at Thu 2022-05-19 12:45:24 UTC, end at Thu 2022-05-19 13:52:04 UTC. -- May 19 13:16:19.848630 dv-cinder-841125-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 13:16:19.860859 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 13:16:19 2022] *** May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: nodename: dv-cinder-841125-6-iscsi May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: machine: x86_64 May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: clock source: unix May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: pcre jit disabled May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: detected number of CPU cores: 4 May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: current working directory: / May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: detected binary path: /usr/bin/uwsgi-core May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: your processes number limit is 47680 May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: your memory page size is 4096 bytes May 19 13:16:19.862984 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: detected max file descriptor number: 2048 May 19 13:16:19.863854 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: lock engine: pthread robust mutexes May 19 13:16:19.863854 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: thunder lock: enabled May 19 13:16:19.863854 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 13:16:19.863854 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 13:16:19.907784 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: Python main interpreter initialized at 0x55b91328c9c0 May 19 13:16:19.907784 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: python threads support enabled May 19 13:16:19.907784 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: your server socket listen backlog is limited to 100 connections May 19 13:16:19.907784 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: your mercy for graceful operations on workers is 90 seconds May 19 13:16:19.908177 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: mapped 403077 bytes (393 KB) for 2 cores May 19 13:16:19.908177 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: *** Operational MODE: preforking *** May 19 13:16:19.908492 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: *** uWSGI is running in multiple interpreter mode *** May 19 13:16:19.908492 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: spawned uWSGI master process (pid: 80237) May 19 13:16:19.908736 dv-cinder-841125-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 13:16:19.909515 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: spawned uWSGI worker 1 (pid: 80248, cores: 1) May 19 13:16:19.909932 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: spawned uWSGI worker 2 (pid: 80249, cores: 1) May 19 13:16:19.909932 dv-cinder-841125-6-iscsi devstack@keystone.service[80237]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 13:16:25.576915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:16:25.576915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: from cryptography.utils import int_from_bytes May 19 13:16:25.577813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:16:25.577813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: from cryptography.utils import int_from_bytes May 19 13:16:25.627461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 13:16:25.627640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 13:16:25.627807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] command line args: [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 13:16:25.628096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 13:16:25.628298 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ================================================================================ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 13:16:25.628536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] admin_token = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.628770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_dir = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.628911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.629112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:16:25.629112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes May 19 13:16:25.629674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_source = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.629674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] debug = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.629799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 13:16:25.629799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: from cryptography.utils import int_from_bytes May 19 13:16:25.629908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.629908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.630127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.630268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.630516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.630745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.630907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_config_append = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.631101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.631281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_dir = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.631429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.631817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.631903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.632157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.632406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.632584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.632758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.632994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.633243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.633520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.633738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.633968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.634232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.634635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.634889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.635168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.635661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.24/identity {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.635844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] publish_errors = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.636076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.636306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.636485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.636729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.636961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.637197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] standard_threads = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.637534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_journal = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_json = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_stderr = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_syslog = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.638846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.639049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.639288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.639569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.640457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.640644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.640888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.641141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.641382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.641659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.641909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.642168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.642405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.642653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.642891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.643139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.643391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.643633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.643890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.644143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.644387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.644624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.644877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.645148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.645402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.645672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.645906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.646136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.646368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.646604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.646855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.647104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.647335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.647567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.647813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.648061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.648299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.648548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.648781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.649020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.649256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.649512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.649746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.649976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.650208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.650444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.650702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.650935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.651170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.651401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.651634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.651869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.652121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.652349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.652586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.652820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.653054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.653287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.653547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.653783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.654034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.654269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.654505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.654737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.655008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.655297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.655531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.655764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.656118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.656396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.656635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.656885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.657131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.657372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.657634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.657871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.658120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.658357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.658596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.658840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.659079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.659318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.659554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.659790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.660027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.660301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.660542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.660784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.661021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.661263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.661524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.661763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.662004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.662233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.662463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.662693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.662928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.663157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.663712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.663949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.664206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.664445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.664826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.665090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.665334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.665594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.external = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.665827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.666126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.666362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.666597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.password = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.666829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.token = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.667071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.667308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.667558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.667790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.668038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.668286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.668519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.668755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.669004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.669251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.669526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.669765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.670010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.670245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.670510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.670779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.671105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.671385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.671690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.671930 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.672195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.672441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.672698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.672937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.673173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.673428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.673695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.673928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.674175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.674412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.674666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.674903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.675158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.675393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.675643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.675880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.676129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.676520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.676636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.676831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.677077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.677309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.677573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.677824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.678077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.678309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.678437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 13:16:25.678524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 13:16:25.678596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.678712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] command line args: [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 13:16:25.678786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.678954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 13:16:25.679033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.679161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ================================================================================ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 13:16:25.679237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.679378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.679462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.679624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_dir = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.679719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.679800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.679916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.679992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] config_source = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.680150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] debug = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.680241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.680364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.680526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.680607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.680679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.680794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.680869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.681006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.681095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.681257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.681336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.681512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.681592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.681735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_config_append = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.681838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.681914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.682011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.682096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_dir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.682199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.682276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.682434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.682516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.682682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.682759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.682927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.683048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.683172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.683253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.683352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.683431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.683530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.683647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.683773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.683878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.684016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.684117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.684248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.684336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.684481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.684578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.684712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.684816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.684945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.685099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.685194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.685272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.685460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.685541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.685739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.685829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.686004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.686234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.686310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.24/identity {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.686495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.686713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] publish_errors = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.687006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.687219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.687300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.687450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.687536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.687699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.687793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.687936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.688014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.688172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] standard_threads = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.688250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.688404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.688493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.688646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.688723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.688827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.688904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.689005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_journal = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.689130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.689207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_json = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.689344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_stderr = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.689444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.689543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] use_syslog = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.689635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.689777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 13:16:25.689879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690771 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.690847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.691966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692771 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.692928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.693973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.694941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.695902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.696996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.697969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.698997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.699926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700302 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.700863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701098 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.701834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.702887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.703954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.704927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.705988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.706934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707426 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.707891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.708991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.709956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.710997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.711949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.712898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.713866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.714981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.external = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.715969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.password = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] auth.token = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.716909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.717005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 13:16:25.717089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.717300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.717555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.717883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.718342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.718614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.718854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.719105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.719355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.719604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.719844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.720089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.720324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.720586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.720840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.721077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.721333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.721644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.721893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.722157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.722402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.722659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.722893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.723131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.723376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.723623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.723854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.724101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.724338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.724589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.724825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.725077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.725313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.725584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.725825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.726071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.726318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.726557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.726793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.727039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.727269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.727506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.727754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.728006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.728239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.728470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.728700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.728944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.729188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.729461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.729718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.729963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.730234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.730473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.730713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.730944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.731182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.731417 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.731655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.731909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.732150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.732398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.732631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.732877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.733122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.733519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.733985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.734327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.734561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.734802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.735035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.735275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.735513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.735758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.735996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.736235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.736484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.736732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.736964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.737197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.737496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.737749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.737986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.738223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.738460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.738692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.738935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.739194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.739426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.739674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.739920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.740164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.740399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.740629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.740863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.741096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.741338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.741591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.741839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.742084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.742329 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.742574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.742817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.743048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.743295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.743531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.743765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.744000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.744233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.744480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.744741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.744977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.745232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.745486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.745733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.745970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.746209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.746445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.746690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.746932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.747166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.747404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.747646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.747893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.748134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.748216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.748369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.748455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.748598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.748679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.748854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.749279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.749762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.750012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.750256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.750494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.750743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.750977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.751229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.751484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.751742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.751991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.752238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.752485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.752730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.752977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.753214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.753485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.753739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.753985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.754230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.754639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.754876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.755122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.755371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.755617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.755866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.756101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.756356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.756610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.756850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.757080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.757312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.757473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.757558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.757811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.758077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.758326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.758565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.758803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.759034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.759271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.759507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.759755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.759992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.760246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.760492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.760731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.760975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.761219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.761495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.761733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.761974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.762211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.762457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.762704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.762936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.763178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.763482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.763741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.763988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.764236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.764472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.764899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.765348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.765675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.765916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.766157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.766409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.766648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.766894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.767131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.767379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.767623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.767869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.768101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.768342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.768578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 13:16:25.768749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80249) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 13:16:25.786019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.786266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.794673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.794910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.795144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.805649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.824437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.824639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.839294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.862895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.889579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.889937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.900702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.900941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.955459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.955657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.955914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.956999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.957225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.979789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.980959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.981107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.981234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.981337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.992664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:25.993188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.006554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.006830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.007030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.007278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.007534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.007751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.007974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80249) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.008112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.008294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.008525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.008748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.008974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.009154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.009394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 13:16:26.025666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.025797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.025865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:16:26.025972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:16:26.026214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.026338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.026414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:16:26.026489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 13:16:26.026679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.026807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.026891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.026969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.027177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.027304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.027414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.027519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.039006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.039282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.039552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.039828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.040797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.041078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.041348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.041652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.047928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.050058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.051211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.051493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.051778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.052170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.052459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.052723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.053000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.053279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.053403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.053627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.053771 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.053899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.054037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.054162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.054291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.054511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.054780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.055038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.055314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.055585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.055853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.056122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.080648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.082313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.082598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.083124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.084733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.085014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.087855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.088136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.088417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.090352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.090545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.090836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.095903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.096182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.098631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.098914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.100764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.101055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.101364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.101618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.101920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.102192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.102456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.102723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.103489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.103775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.104058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.104342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.104600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.104877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.105141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.105405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.121396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.121669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.125778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.126057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.126288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.129907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.130206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.130551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.134157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.134455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.135136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.138736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.139405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.142589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.142887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.143173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.143362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.143460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.143724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.144003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.147076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.147379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.147660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.147944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.148232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.148509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.162835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.163511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.163793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.164068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.168393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.168773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.169046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.169323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.172106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.173822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.174108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.177153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.177442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.177662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.177763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.177987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.178281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.179405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.179686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.182741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.183022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.183311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.183612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.183886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.191597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.191877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.192157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.192438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.192760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.192977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.193279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.197672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.197951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.198249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.198531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.198808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.199078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.199351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.211405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.214666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.217841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.217969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.219581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.221227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.221352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.221587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.221815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.224680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.226314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.227964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.228253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.228533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.229131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.230785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.234129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.234416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.234711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.234988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.236375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.238020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.241227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.241549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.241831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.242103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.244141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.244484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.244807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.251650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.251937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.252189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.252848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.253139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.253447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.253722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.253995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.254277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.254564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.254837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.255102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.255370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.255638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.257982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.257982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.257982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.260055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80249) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 13:16:26.260324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.260601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.260889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.261170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.261461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.261753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.262022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.262291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.262565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.262842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.263107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.263367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.263639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80249) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 13:16:26.295277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b91328c9c0 pid: 80248 (default app) May 19 13:16:26.298769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4383-26f5-40ca-a186-5daa101e46a0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:26.299068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4383-26f5-40ca-a186-5daa101e46a0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:26.299349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a4383-26f5-40ca-a186-5daa101e46a0 None None] PATH_INFO: `/v3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:26.303375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b91328c9c0 pid: 80249 (default app) May 19 13:16:26.303540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1/1] 10.222.0.24 () {50 vars in 791 bytes} [Thu May 19 13:16:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:34.615203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:34.616052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:34.616052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:34.623936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1/2] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:34.632409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:34.633105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:34.633256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:34.708834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: INFO dbcounter [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] Registered counter for database keystone May 19 13:16:34.734606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG oslo_db.sqlalchemy.engines [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 13:16:34.743472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writer thread running {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 13:16:34.922780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG passlib.handlers.bcrypt [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80248) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 13:16:34.923113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG passlib.handlers.bcrypt [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80248) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 13:16:34.979142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:34.980739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:35.109007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 2/3] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 13:16:35.157066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:35.226396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: INFO dbcounter [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] Registered counter for database keystone May 19 13:16:35.248233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG oslo_db.sqlalchemy.engines [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80249) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 13:16:35.258116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writer thread running {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 13:16:35.385603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b2a7b-5e80-48e0-b7bf-1696a0bfefae None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:35.390857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:35.391552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:35.391840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:35.392122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:35.401388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:35.454523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 13:16:35.454865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac381579-75dd-4f8a-ba9b-0faa903993c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:35.466000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 2/4] 10.222.0.24 () {58 vars in 1324 bytes} [Thu May 19 13:16:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:16:35.472339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:35.487508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12f4b6-e92c-48ff-a1fc-6f7ca2fd42c8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:35.492280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:35.493450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:35.493748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:35.494031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] PATH_INFO: `/v3/projects/863bcb4c192047ed8ae210a081ac829f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:35.495584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] RBAC: Authorizing `identity:get_project(project_id=863bcb4c192047ed8ae210a081ac829f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:35.659108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 13:16:35.659749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5eb70-fca0-46f2-93cd-e35adb4800d4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:35.664553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 3/5] 10.222.0.24 () {56 vars in 1207 bytes} [Thu May 19 13:16:35 2022] GET /identity/v3/projects/863bcb4c192047ed8ae210a081ac829f => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:37.856553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6816e-8f68-45fc-86e3-19115d8cecb9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:37.857461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6816e-8f68-45fc-86e3-19115d8cecb9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:37.857461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6816e-8f68-45fc-86e3-19115d8cecb9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:37.864372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 3/6] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:37.873355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:37.873820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:37.873964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:37.899648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:37.901160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:37.911424 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 4/7] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 13:16:37.934591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6816e-8f68-45fc-86e3-19115d8cecb9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:37.956233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6816e-8f68-45fc-86e3-19115d8cecb9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:37.962289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a7709-4f69-431c-8456-5086b4dee3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:37.962927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a7709-4f69-431c-8456-5086b4dee3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:37.963213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a7709-4f69-431c-8456-5086b4dee3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:37.963476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a7709-4f69-431c-8456-5086b4dee3b1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:37.972335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a7709-4f69-431c-8456-5086b4dee3b1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:37.974160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a7709-4f69-431c-8456-5086b4dee3b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:37.983909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 4/8] 10.222.0.24 () {58 vars in 1324 bytes} [Thu May 19 13:16:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:16:37.992595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:38.008055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8306c-9186-46d0-bb63-efe9238837cd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:38.016157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082de14-8b08-4541-9334-039fd8a98fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:38.017036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082de14-8b08-4541-9334-039fd8a98fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:38.017319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082de14-8b08-4541-9334-039fd8a98fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:38.017636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082de14-8b08-4541-9334-039fd8a98fbe admin admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:38.019200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082de14-8b08-4541-9334-039fd8a98fbe admin admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:38.021055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082de14-8b08-4541-9334-039fd8a98fbe admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:38.022682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 5/9] 10.222.0.24 () {56 vars in 1198 bytes} [Thu May 19 13:16:37 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:40.267386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e0842-3e08-4483-9da3-84f2bb8fd4e8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.267882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e0842-3e08-4483-9da3-84f2bb8fd4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.267974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e0842-3e08-4483-9da3-84f2bb8fd4e8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.269322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 5/10] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:40.275003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.275468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.275552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.292783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:40.293978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:40.307606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a72c1-839b-46e4-a334-8dcfb09ff1b8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.307971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a72c1-839b-46e4-a334-8dcfb09ff1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.308249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a72c1-839b-46e4-a334-8dcfb09ff1b8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.309643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 6/11] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:40.314708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.314992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.315264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.354466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80249) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 13:16:40.354887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG passlib.handlers.bcrypt [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80249) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 13:16:40.373530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 6/12] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:16:40.397166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:40.400819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:40.402454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:40.423370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:40.502814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 7/13] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:40.506279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bca2c-46da-47cf-b78a-3e1aaafbfba0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:40.524568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:40.542414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:40.543220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.543499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.543776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.555605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:40.566893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:40.567917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:16:40.568768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 7/14] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:40.573858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:40.625744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3135d00-78e1-461c-97f0-72ee0493a618 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:40.628032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:40.654350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:40.665222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:40.666263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.666551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.666825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.674331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831529c-448b-418c-bf91-aaf1997387b4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:40.709706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:40.710384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.710666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.710943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.711708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:40.712012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:40.713490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:40.726015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 8/15] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:40.731359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:40.737066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:40.738657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:40.827572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ccc08-64a9-47d2-8238-84520c107777 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:40.835121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 8/16] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:16:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:16:40.840566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:40.843826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:40.862302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:40.863218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.863490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.863769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.865013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:40.866522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:40.868203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 9/17] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:40 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:40.873584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:40.888802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:40.942999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47553279-b6b6-4539-8e1e-67220a770a3a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:40.973851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02972c-14fa-4df5-8e80-3bd43aec417b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:40.977947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:40.978755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:40.979031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:40.979309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:40.997613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:40.999156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:41.009153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:41.010033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:41.010316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:41.010591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:41.011880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:41.013356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:41.015363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 10/18] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:16:41.020299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:41.021210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:41.022078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:16:41.022078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:16:41.022078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:16:41.022078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:16:41.022303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:41.022378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 9/19] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:16:41.028333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:41.084249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:41.104972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:41.123430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ab153-466e-4430-8977-aa44531973a7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:41.128138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0cdb0-b8d6-4506-9b64-7f0c00476d22 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:41.160251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:41.160956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:41.161234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:41.161543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:41.162365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.domain.id=default, user.id=e50c121ca4544e98a148c3b6b1759f0d) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:41.162617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:41.162740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:41.163598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:41.163880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:41.164157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:41.164460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf85a78-0496-40c0-b2a7-b821df39c48a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:41.164887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:41.165227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:41.166652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525c200-28e4-4168-82fb-ac38a35454c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:41.171415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 11/20] 10.222.0.24 () {56 vars in 1322 bytes} [Thu May 19 13:16:41 2022] GET /identity/v3/role_assignments?user.id=e50c121ca4544e98a148c3b6b1759f0d&scope.domain.id=default&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:41.187777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 10/21] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:43.599769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f790-7c7b-4b08-8e8b-8e78b64fabb0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:43.600770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f790-7c7b-4b08-8e8b-8e78b64fabb0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:43.601257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f790-7c7b-4b08-8e8b-8e78b64fabb0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:43.602844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 12/22] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:43.608532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:43.609028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:43.609282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:43.704614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c75f-a719-4910-bd8e-c3064a2ba717 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:43.705250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c75f-a719-4910-bd8e-c3064a2ba717 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:43.705794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c75f-a719-4910-bd8e-c3064a2ba717 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:43.707719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 13/23] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:43.713293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:43.713961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:43.714443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:43.750776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:43.752362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:43.767597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:43.769141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:43.915854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 11/24] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:43.947752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 14/25] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:16:43.967309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:43.982383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:44.020729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:44.082396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8d935-727a-4d38-9c19-0b687cd3ecef None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:44.126153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:44.127166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.127585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.128013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.152772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:44.154294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:44.175656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:16:44.197605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:44.198533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:44.199848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:16:44.199848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:16:44.199848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:16:44.199848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:16:44.199848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 12/26] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:44.207751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:44.247609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad544ba-94e8-4d43-9718-0127be1842ef None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:44.347065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:44.347884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.348174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.348456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.373797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:44.382265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:44.383131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:44.383940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:16:44.386478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 15/27] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:16:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:16:44.392364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:44.415944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:44.419688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fbfd2-64c7-44b8-bb28-0f6e9cd1eab4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:44.488662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:44.489336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.489657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.489935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.490679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:44.490986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:44.492379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dda371-949a-4404-a6a9-10a759c663dc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:44.507011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 13/28] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:44.508582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20820-aa30-4e17-baf5-1f191d533e3c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.509011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20820-aa30-4e17-baf5-1f191d533e3c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.509430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d20820-aa30-4e17-baf5-1f191d533e3c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.510789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 14/29] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:44.515688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20820-aa30-4e17-baf5-1f191d533e3c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:44.557243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94db849-7195-4b42-b1c9-582b929b1b71 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:44.594994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:44.595966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.596496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.596945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.597942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:44.598499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:44.600191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e03ef6-8bdb-4322-bdca-1c38c8596912 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:44.602297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:44.616307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 16/30] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:44.618170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cfbf4-df53-416c-8974-6bce6d900160 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.618639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cfbf4-df53-416c-8974-6bce6d900160 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.619091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cfbf4-df53-416c-8974-6bce6d900160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.624173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d20820-aa30-4e17-baf5-1f191d533e3c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:44.636995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d17cfbf4-df53-416c-8974-6bce6d900160 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:44.638557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d17cfbf4-df53-416c-8974-6bce6d900160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:44.661490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80f4f5-0de5-454d-8931-66fe4b4f3add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:44.662215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f4f5-0de5-454d-8931-66fe4b4f3add None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.662492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f4f5-0de5-454d-8931-66fe4b4f3add None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.662950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80f4f5-0de5-454d-8931-66fe4b4f3add None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.664606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80f4f5-0de5-454d-8931-66fe4b4f3add None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:44.666419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80f4f5-0de5-454d-8931-66fe4b4f3add None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:44.709978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 17/31] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:44.711643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800c56e-5f1c-4454-875f-c32d13103b09 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.712161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800c56e-5f1c-4454-875f-c32d13103b09 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.712646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800c56e-5f1c-4454-875f-c32d13103b09 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.714140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 18/32] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:44.715025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800c56e-5f1c-4454-875f-c32d13103b09 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:44.819732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800c56e-5f1c-4454-875f-c32d13103b09 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:44.833801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 15/33] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:16:44 2022] POST /identity/v3/projects => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:16:44.835642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.836090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.836519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.849772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:44.858336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:44.858527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:44.859618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:44.860079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:44.860373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:44.860710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:44.862515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:44.864276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:44.864407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:44.866460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 19/34] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:44 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:44.872240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:45.004794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 16/35] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:45.005727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:45.062356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd53e5-d01a-4262-a791-287791fb88ce None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:45.109101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:45.110152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.110674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.111155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.112109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:45.113723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:45.153866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75edb04-adac-407f-adc9-641f8ef88af6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:45.189550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:45.190624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.191054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.191472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.192811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:45.194658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:45.196458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 17/36] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:45.201555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:45.212879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:45.268568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 20/37] 10.222.0.24 () {60 vars in 1131 bytes} [Thu May 19 13:16:44 2022] POST /identity/v3/roles => generated 225 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:16:45.273501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:45.333229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7f493-1d92-4a40-ad80-381774ea2791 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:45.363074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:45.388391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6469d64-eee9-4bee-b2d4-c0707d7593a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:45.389767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6469d64-eee9-4bee-b2d4-c0707d7593a7 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.390471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6469d64-eee9-4bee-b2d4-c0707d7593a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.390952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6469d64-eee9-4bee-b2d4-c0707d7593a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.391939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6469d64-eee9-4bee-b2d4-c0707d7593a7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:45.393658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6469d64-eee9-4bee-b2d4-c0707d7593a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:45.412250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 18/38] 10.222.0.24 () {60 vars in 1131 bytes} [Thu May 19 13:16:45 2022] POST /identity/v3/roles => generated 219 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:16:45.414583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf425ed3-0e9a-42b2-a343-fa24730b70ed None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.415142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf425ed3-0e9a-42b2-a343-fa24730b70ed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.415635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf425ed3-0e9a-42b2-a343-fa24730b70ed None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.417730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 19/39] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:45.426656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87ecca-6852-45ae-b72f-7705991d1122 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:45.431129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a4e6-79a6-488e-8c21-127e9fdaceb9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.431395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a4e6-79a6-488e-8c21-127e9fdaceb9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.431683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a4e6-79a6-488e-8c21-127e9fdaceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.463927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c317a4e6-79a6-488e-8c21-127e9fdaceb9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:45.465155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c317a4e6-79a6-488e-8c21-127e9fdaceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:45.478893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:45.480219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.480711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.481350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] PATH_INFO: `/v3/domains/default/users/e50c121ca4544e98a148c3b6b1759f0d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.486506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e50c121ca4544e98a148c3b6b1759f0d, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:45.488411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:45.508689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 21/40] 10.222.0.24 () {58 vars in 1361 bytes} [Thu May 19 13:16:45 2022] PUT /identity/v3/domains/default/users/e50c121ca4544e98a148c3b6b1759f0d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:16:45.556422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:45.561477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 20/41] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:45.596951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:45.629543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0043c8-9a99-46b2-ac63-5c64f233dd83 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.630057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0043c8-9a99-46b2-ac63-5c64f233dd83 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.630057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0043c8-9a99-46b2-ac63-5c64f233dd83 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.631707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 21/42] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:45.638632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.639321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.639829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.662829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:45.664828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:45.740099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe36a85-1ac0-4c34-9fa0-53bd4d36a97c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:45.754477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:45.755533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 22/43] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:16:45.776782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:45.777497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.777805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.778077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.778839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:45.779428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:45.780320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:45.795912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 22/44] 10.222.0.24 () {60 vars in 1131 bytes} [Thu May 19 13:16:45 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:16:45.895002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93887b-2fa0-4612-be14-93c8607825d3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:45.925943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:45.936048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908dd93-5d13-4be6-95b5-f2545f3cf846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:45.936938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908dd93-5d13-4be6-95b5-f2545f3cf846 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:45.937359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908dd93-5d13-4be6-95b5-f2545f3cf846 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:45.937674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908dd93-5d13-4be6-95b5-f2545f3cf846 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:45.938968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908dd93-5d13-4be6-95b5-f2545f3cf846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:45.940528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908dd93-5d13-4be6-95b5-f2545f3cf846 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:45.947191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 23/45] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:45.948192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:46.057746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af00ed4-2bec-4617-b7db-408297ce07d6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:46.091697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38bf0b-92b7-445b-b159-7e1c0fb33973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:46.092574 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38bf0b-92b7-445b-b159-7e1c0fb33973 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:46.092862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38bf0b-92b7-445b-b159-7e1c0fb33973 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:46.093143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38bf0b-92b7-445b-b159-7e1c0fb33973 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:46.093897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38bf0b-92b7-445b-b159-7e1c0fb33973 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:46.095351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38bf0b-92b7-445b-b159-7e1c0fb33973 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:46.123774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 23/46] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:16:45 2022] POST /identity/v3/projects => generated 312 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:16:47.790541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891427b9-8c7e-4068-bf34-3c06eb1b866c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:47.791085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891427b9-8c7e-4068-bf34-3c06eb1b866c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:47.791085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891427b9-8c7e-4068-bf34-3c06eb1b866c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:47.792487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 24/47] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:47.798295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:47.798711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:47.798836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:47.817058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:47.818619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:47.847352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:47.898180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 24/48] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:47.921125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891427b9-8c7e-4068-bf34-3c06eb1b866c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.027427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891427b9-8c7e-4068-bf34-3c06eb1b866c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.062999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.063724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.064005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.064281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.075917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:48.084684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:48.085621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:16:48.086470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 25/49] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:16:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:48.091793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.196579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a1c4c-0c2b-40c5-8a14-789d3f232698 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.234485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.235160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.235442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.235718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.236483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:48.236789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:48.238239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55419925-f925-4ba1-802b-72642b8d4828 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:48.248353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 25/50] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:48.254341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.313749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:48.362515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee7f4-1d1d-4a06-875b-c83d53ab3727 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.386749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864be17-5f62-458f-a7ee-76d535a6905c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.387041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864be17-5f62-458f-a7ee-76d535a6905c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.387327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864be17-5f62-458f-a7ee-76d535a6905c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.388702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 26/51] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:48.394073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.394369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.394657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.397366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.398280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.398564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.398841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.400314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:48.401877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:48.403519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 26/52] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:48 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:48.409816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.412641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:48.414151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:48.428197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:48.484503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 27/53] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:48.506919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.509341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50865a48-7788-4b8a-8340-bae6475f2f3e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.534435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:48.546516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.547318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.547604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.547888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.549096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:48.550600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:48.552244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 27/54] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:48.557938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.607889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd60a25-c0cf-4f41-b490-c68278bba9f5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.641882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.642685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.642965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.643269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.644494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:48.646006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:48.647642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 28/55] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:48.653289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:48.657648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bda567-49a1-4b6d-82dc-1db96ce895b1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.671476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:48.692012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.692676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.693056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.693341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.694221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.domain.id=default, user.id=e50c121ca4544e98a148c3b6b1759f0d) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:48.694539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:48.695998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8bf43-7b19-4736-b337-b8b9771c4651 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:48.705113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 28/56] 10.222.0.24 () {56 vars in 1322 bytes} [Thu May 19 13:16:48 2022] GET /identity/v3/role_assignments?user.id=e50c121ca4544e98a148c3b6b1759f0d&scope.domain.id=default&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:48.769138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b3bb2-0c50-4c15-a5e6-3de2f2e8e7fc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:48.807464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd30fe-1558-427f-9893-6c6b3afe3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:48.808169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd30fe-1558-427f-9893-6c6b3afe3d28 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:48.808422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd30fe-1558-427f-9893-6c6b3afe3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:48.808702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd30fe-1558-427f-9893-6c6b3afe3d28 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:48.809520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd30fe-1558-427f-9893-6c6b3afe3d28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:48.810964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd30fe-1558-427f-9893-6c6b3afe3d28 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:48.849450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 29/57] 10.222.0.24 () {60 vars in 1140 bytes} [Thu May 19 13:16:48 2022] POST /identity/v3/projects => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:16:51.082503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb840-bc4f-45aa-8873-b91fea903cd9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:51.083090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb840-bc4f-45aa-8873-b91fea903cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:51.083090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb840-bc4f-45aa-8873-b91fea903cd9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:51.084446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 29/58] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:51.090476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:51.090893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:51.091050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:51.110748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:51.111941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:51.181076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:51.188501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 30/59] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:51.218529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb840-bc4f-45aa-8873-b91fea903cd9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:51.282946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:51.333049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb840-bc4f-45aa-8873-b91fea903cd9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:51.370618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb954d-37d6-41f9-acbf-61ae3bac186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:51.371770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb954d-37d6-41f9-acbf-61ae3bac186f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:51.372187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb954d-37d6-41f9-acbf-61ae3bac186f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:51.372668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb954d-37d6-41f9-acbf-61ae3bac186f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:51.374382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb954d-37d6-41f9-acbf-61ae3bac186f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:51.376630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb954d-37d6-41f9-acbf-61ae3bac186f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:51.379389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 30/60] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:51.388770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:51.503826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e9525-6c22-445c-9077-3e52d24b7660 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:51.539865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48658e-1416-447e-899b-47e513aeddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:51.540533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48658e-1416-447e-899b-47e513aeddef None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:51.540818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48658e-1416-447e-899b-47e513aeddef None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:51.541095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48658e-1416-447e-899b-47e513aeddef None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:51.541841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48658e-1416-447e-899b-47e513aeddef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:51.543316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48658e-1416-447e-899b-47e513aeddef None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:51.571500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 31/61] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:16:51 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:16:54.917297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c472835-4fbc-4419-88d1-061b04422630 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:54.917848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c472835-4fbc-4419-88d1-061b04422630 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:54.917848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c472835-4fbc-4419-88d1-061b04422630 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:54.919234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 31/62] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:54.924837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ff754-5f7a-4e06-a983-0809a121931b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:54.925741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ff754-5f7a-4e06-a983-0809a121931b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:54.926216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ff754-5f7a-4e06-a983-0809a121931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:54.947615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9a9ff754-5f7a-4e06-a983-0809a121931b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:54.949136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9a9ff754-5f7a-4e06-a983-0809a121931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:55.014289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.037843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 32/63] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:16:55.060513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c570327-7849-4913-8dbf-3bbff85fccb4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.060871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c570327-7849-4913-8dbf-3bbff85fccb4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.061318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c570327-7849-4913-8dbf-3bbff85fccb4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.062873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 32/64] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:55.068398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.068699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.068985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.075789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c570327-7849-4913-8dbf-3bbff85fccb4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.096439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:55.097696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:55.214081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 33/65] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:16:55.263622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.267179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c570327-7849-4913-8dbf-3bbff85fccb4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.302610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.315824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa9fc9-c7e3-4f8b-8250-d532919b5188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.316637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa9fc9-c7e3-4f8b-8250-d532919b5188 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.316922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa9fc9-c7e3-4f8b-8250-d532919b5188 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.317211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa9fc9-c7e3-4f8b-8250-d532919b5188 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.331365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa9fc9-c7e3-4f8b-8250-d532919b5188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1daa9fc9-c7e3-4f8b-8250-d532919b5188 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:55.342333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:55.343186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:16:55.343960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 33/66] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:55.345227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6a2a9-4ce6-4cb1-a42c-07131fd6ae72 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.345559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6a2a9-4ce6-4cb1-a42c-07131fd6ae72 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.345842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6a2a9-4ce6-4cb1-a42c-07131fd6ae72 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.347105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 34/67] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:55.349174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6a2a9-4ce6-4cb1-a42c-07131fd6ae72 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.396017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa51e16-d60b-4919-9539-dcabc4df11d0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.417093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.445668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39ff77-a405-4951-b1ab-5337e184c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.447631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ff77-a405-4951-b1ab-5337e184c887 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.448141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ff77-a405-4951-b1ab-5337e184c887 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.448628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ff77-a405-4951-b1ab-5337e184c887 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.463674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39ff77-a405-4951-b1ab-5337e184c887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ae39ff77-a405-4951-b1ab-5337e184c887 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:55.474838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:55.475755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:16:55.476536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:16:55.479048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 34/68] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:16:55.480264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.480555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.480838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.496493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6a2a9-4ce6-4cb1-a42c-07131fd6ae72 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.507472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:55.509669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:55.538183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.539169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.539854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.540307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.541767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:55.542321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:55.544474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:55.559862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 35/69] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:55.561642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.592690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 35/70] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:55.593948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.621652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.676005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e19a56-ee44-4bf1-8965-8d2ca91544d8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.708596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731221b9-33e5-4047-b840-5e15a6748bfe None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.709286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.710002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.710295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.710579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.711068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.711348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:55.711664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:55.713188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf543b-c657-49ce-ba89-83b4f89dfee7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:55.722024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 36/71] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:55.723277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7f3c-2fce-4ef6-9965-7eff00f8731c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.723571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7f3c-2fce-4ef6-9965-7eff00f8731c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.723861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7f3c-2fce-4ef6-9965-7eff00f8731c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.727003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 37/72] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:55.729892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7f3c-2fce-4ef6-9965-7eff00f8731c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.745023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519e48a-5de0-46a4-bcd8-aaf79e04c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.745941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519e48a-5de0-46a4-bcd8-aaf79e04c69d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.746225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519e48a-5de0-46a4-bcd8-aaf79e04c69d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.746499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519e48a-5de0-46a4-bcd8-aaf79e04c69d None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.760296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519e48a-5de0-46a4-bcd8-aaf79e04c69d None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:55.761813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519e48a-5de0-46a4-bcd8-aaf79e04c69d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:55.763521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 36/73] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:55.767691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecef91-8595-4869-ab61-f7e01a78bb2a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.768202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecef91-8595-4869-ab61-f7e01a78bb2a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.768310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecef91-8595-4869-ab61-f7e01a78bb2a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.769573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 37/74] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:55.769573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecef91-8595-4869-ab61-f7e01a78bb2a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.832147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7f3c-2fce-4ef6-9965-7eff00f8731c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.840861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.867469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b67e74-17fe-4e16-996b-b6aa2f70f1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.868286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b67e74-17fe-4e16-996b-b6aa2f70f1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.868572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b67e74-17fe-4e16-996b-b6aa2f70f1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.868849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b67e74-17fe-4e16-996b-b6aa2f70f1a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.871251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecef91-8595-4869-ab61-f7e01a78bb2a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:55.879589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b67e74-17fe-4e16-996b-b6aa2f70f1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-63b67e74-17fe-4e16-996b-b6aa2f70f1a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:55.887491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:55.888504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:16:55.890026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 38/75] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:55.890680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e250c-ce71-4728-a269-c073ca2adad9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.890680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e250c-ce71-4728-a269-c073ca2adad9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.890680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e250c-ce71-4728-a269-c073ca2adad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.908260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:55.908447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c8e250c-ce71-4728-a269-c073ca2adad9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:55.909267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.909629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c8e250c-ce71-4728-a269-c073ca2adad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:55.909787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.910027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:55.911333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:55.913116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:55.915097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 38/76] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:55.915861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:55.947606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:55.978302 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 39/77] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:55.979527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:55.979846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:55.980107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.005065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:56.006286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:56.018395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de42e3-2fa1-449a-873c-7158d947b876 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.054064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.054724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.055021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.055306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.073637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 40/78] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:56.074472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.103662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.131953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.133500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.163946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.174599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a205-8051-41b3-92b6-28c5f28ce118 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.208126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.208783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.209065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.209344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.210127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:56.210438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.211828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.219577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 41/79] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:56.219577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.236621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 39/80] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:16:56.236621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.319932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.329815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee00da8-1f15-4dd4-88cf-399f84d7d733 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.334522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde836c3-4270-48d2-a5f7-6cf075a866f3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.361870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.362519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.362805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.362941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.363106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.369106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.369935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.370223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.370499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.380848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:56.388478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:56.390157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 40/81] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:56.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.436398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.437866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.488556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488af5b-758c-4f43-917e-9ae06d636a48 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.521500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.522288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.522572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.522850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.524019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.524783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.525512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.527212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 41/82] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:56.528048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.539482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 42/83] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:16:56.540171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.588647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.629395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088650fd-ee1f-4806-bb87-ec516970ac5a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.639575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41807a1-12d6-4307-b257-58451a3811cf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.664534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.665437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.665728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.666008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.667287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.668751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.670471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 42/84] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:56.671229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.673205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.674103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.674388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.674699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.675986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.677456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.683871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 43/85] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:56.683871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.739217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.771005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5d3c9-a5e5-4124-be4f-c24c522ef319 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.778730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91685092-6932-456b-8e5b-ed0f03fc72a7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.796671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.805558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.806228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.806512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.806789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.807544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:56.807857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.809238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.812935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.813636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.813925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.814205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.814920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.816348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.817435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 43/86] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:56.818229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.843945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 44/87] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:16:56 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:16:56.844776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:56.943190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b228124-97ba-4a09-b2bb-a41b6a3d2925 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.961032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219933e1-f3c5-44d4-bb3b-bc65b5bc09c6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:56.977221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:56.981052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:56.982102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:56.982411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:56.982830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:56.984379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:56.985966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:56.988173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 44/88] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:56.994087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.002595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.003285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.003548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.004443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.058308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:57.089605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.091268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.115789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b249a-8cf2-4418-bf84-afe20f2ab6be None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.161152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.161859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.162243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.162430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.163591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=b663cda428a149fa9a7a5c13fa738c8d) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:57.163914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.165400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.171772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 45/89] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/role_assignments?user.id=b663cda428a149fa9a7a5c13fa738c8d&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:57.172608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.223751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 45/90] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:16:57.224536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.250507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:57.329035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba0f7b-7b01-4392-9de9-b47f2e8f2d4c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.349819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e47a38-a5f3-45dd-aa70-ac3d931deb03 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.355908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:57.374087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.374990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.375277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.375561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.376861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.379007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.381882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 46/91] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:57.382840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.394162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.395195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.395829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.396321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.397572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=e50c121ca4544e98a148c3b6b1759f0d) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:57.399204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.401121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.409327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 46/92] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/role_assignments?user.id=e50c121ca4544e98a148c3b6b1759f0d&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:57.410751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.537913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:57.541628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf9e78-5dfb-47b0-9f5c-04d6281549b5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.582100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.583410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.584318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.584822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.611431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719ae3d-ae52-47bc-b296-b09069ecba2f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.663732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.665245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.714149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.715042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.715333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.715618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.717010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.718553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.720318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 47/93] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:57.726216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.750569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:57.790649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 47/94] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:16:57.798647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:57.799737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:57.870750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91e487-d28f-4550-9ff1-a00a13e03520 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.943822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac40417-318d-4f91-b942-ef20d47ade8a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:57.958135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.959104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.959562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.960007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.961046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] RBAC: Adding query filter params (role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=b663cda428a149fa9a7a5c13fa738c8d) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:57.961603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.963253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.975746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 48/95] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/role_assignments?user.id=b663cda428a149fa9a7a5c13fa738c8d&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:57.984446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e03cdf-ed07-444c-824a-9e0a3c384055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:57.985444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e03cdf-ed07-444c-824a-9e0a3c384055 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:57.986165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e03cdf-ed07-444c-824a-9e0a3c384055 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:57.986466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e03cdf-ed07-444c-824a-9e0a3c384055 None admin] PATH_INFO: `/v3/projects/4146fddb1e44491196b3cb98804479c4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:57.987902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e03cdf-ed07-444c-824a-9e0a3c384055 None admin] RBAC: Authorizing `identity:get_project(project_id=4146fddb1e44491196b3cb98804479c4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:57.989605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e03cdf-ed07-444c-824a-9e0a3c384055 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:57.992652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 48/96] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/projects/4146fddb1e44491196b3cb98804479c4 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:16:57.997737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:58.149947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3497-6c7e-4bec-98ad-068bcee9079d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:58.183487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:58.201291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:58.202389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:58.202890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:58.203357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:58.204390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=4146fddb1e44491196b3cb98804479c4, user.id=b663cda428a149fa9a7a5c13fa738c8d) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:58.204896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:58.206578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b817f3-fc16-48ca-87d9-e29e60aede23 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:58.218089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 49/97] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:16:57 2022] GET /identity/v3/role_assignments?user.id=b663cda428a149fa9a7a5c13fa738c8d&scope.project.id=4146fddb1e44491196b3cb98804479c4&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:16:59.512524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed67a6-68ca-491b-ae8a-478b7d726edc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:59.513046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed67a6-68ca-491b-ae8a-478b7d726edc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:59.513046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed67a6-68ca-491b-ae8a-478b7d726edc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:59.515166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 49/98] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:16:59.520860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:59.521323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:59.521523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:59.539721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:16:59.540926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:16:59.614409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 50/99] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:16:59.650344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed67a6-68ca-491b-ae8a-478b7d726edc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:59.702624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:59.763339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed67a6-68ca-491b-ae8a-478b7d726edc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:59.803317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:59.804271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:59.804566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:59.804850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:59.817131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:16:59.824979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:16:59.827587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:16:59.833210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 50/100] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:16:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:16:59.834825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:16:59.942003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39733b9b-6536-4140-ad5d-4bfcab97bf24 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:16:59.957691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:16:59.981372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:16:59.982085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:16:59.982378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:16:59.982907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:16:59.984007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:16:59.984325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:16:59.985772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:16:59.994878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 51/101] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:16:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:00.000799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:00.121892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0d6ab-e1aa-4509-83fc-2bf4c96aa3dc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:00.129743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:00.159618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a706-1c0d-4254-bafc-8ff4924848e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:00.160522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a706-1c0d-4254-bafc-8ff4924848e2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.160807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a706-1c0d-4254-bafc-8ff4924848e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.161090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a706-1c0d-4254-bafc-8ff4924848e2 None admin] PATH_INFO: `/v3/users/00b23692a87e493a80a5261b70ab63c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.178742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504a706-1c0d-4254-bafc-8ff4924848e2 None admin] RBAC: Authorizing `identity:get_user(user_id=00b23692a87e493a80a5261b70ab63c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:00.180379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504a706-1c0d-4254-bafc-8ff4924848e2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:00.182352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 51/102] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:16:59 2022] GET /identity/v3/users/00b23692a87e493a80a5261b70ab63c9 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:00.188805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:00.341600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b246b04-b144-485f-a9e9-f80e6b465dc5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:00.387080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:00.387665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.387953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.388231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.451569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:00.483840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:00.485350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:00.503524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09756ff1-d3f8-43b7-ac9d-a16bf1046f41 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.504150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09756ff1-d3f8-43b7-ac9d-a16bf1046f41 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.504662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09756ff1-d3f8-43b7-ac9d-a16bf1046f41 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.506523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 52/103] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:00.511490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.512020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.512506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.542128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:00.544125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:00.660398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 53/104] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:00.696478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 52/105] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:00.698330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:00.702808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:00.821453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:00.829219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb750780-6324-4610-b0b9-04ad7e83309b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:00.873039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77165feb-3111-4e22-90a3-4fbe9ca24cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:00.874469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77165feb-3111-4e22-90a3-4fbe9ca24cac None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.874977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77165feb-3111-4e22-90a3-4fbe9ca24cac None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.875447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77165feb-3111-4e22-90a3-4fbe9ca24cac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.881144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:00.896159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77165feb-3111-4e22-90a3-4fbe9ca24cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-77165feb-3111-4e22-90a3-4fbe9ca24cac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:00.905677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:00.906635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:00.907503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 54/106] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:00.908700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc08af-e88a-4961-ad82-a5f8b321434f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.913632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc08af-e88a-4961-ad82-a5f8b321434f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.913632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc08af-e88a-4961-ad82-a5f8b321434f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.913632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 55/107] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:00.913901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc08af-e88a-4961-ad82-a5f8b321434f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:00.940592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad0fd5-f551-4900-b00e-450d9e88ec62 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:00.982624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8087d57e-e9ba-4e50-affa-bad354d59684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:00.983721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087d57e-e9ba-4e50-affa-bad354d59684 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:00.984081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087d57e-e9ba-4e50-affa-bad354d59684 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:00.984370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087d57e-e9ba-4e50-affa-bad354d59684 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:00.990058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087d57e-e9ba-4e50-affa-bad354d59684 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:00.991549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8087d57e-e9ba-4e50-affa-bad354d59684 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:00.993528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 53/108] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:00 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:01.000100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.000400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.000692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.038834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:01.040060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:01.116467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc08af-e88a-4961-ad82-a5f8b321434f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.153846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.154655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.155030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.155450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.156575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:01.156897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.157434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 54/109] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:01.158357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15011cb1-abe1-423a-9ad0-b34751b6f108 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.158475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.169127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 56/110] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:01.171677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1905c9-1b2a-4777-b948-0f25f6929013 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.172044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1905c9-1b2a-4777-b948-0f25f6929013 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.172694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1905c9-1b2a-4777-b948-0f25f6929013 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.174451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 57/111] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:01.175442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1905c9-1b2a-4777-b948-0f25f6929013 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.198621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:01.260956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:01.288993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afbf35-d532-4058-a7a0-e0b19b684134 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.300205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1905c9-1b2a-4777-b948-0f25f6929013 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.326266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.326936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.327228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.327613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.328468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] RBAC: Adding query filter params (role.id=6091ae42ec74461bbd8177eb121c6d63, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=00b23692a87e493a80a5261b70ab63c9) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:01.328879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.330493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61406f7e-0d74-4421-903a-a228f577cab5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.338870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.339852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.340194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.340475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.341541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 55/112] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:01 2022] GET /identity/v3/role_assignments?user.id=00b23692a87e493a80a5261b70ab63c9&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=6091ae42ec74461bbd8177eb121c6d63 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:01.342353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.343459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.343760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.344045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.344240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.346146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 58/113] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:01 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:01.346977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.378054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:01.379307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:01.454836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 56/114] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:01.456146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.456297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:01.468711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039684-0c02-4133-b5fb-37f012e82c74 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.510842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602073b7-c2b5-4575-9ae5-3b9759b25a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.512063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602073b7-c2b5-4575-9ae5-3b9759b25a78 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.512654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602073b7-c2b5-4575-9ae5-3b9759b25a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.513143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602073b7-c2b5-4575-9ae5-3b9759b25a78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.515677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602073b7-c2b5-4575-9ae5-3b9759b25a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.517867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602073b7-c2b5-4575-9ae5-3b9759b25a78 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.520805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 59/115] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:01.522910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d32f6-d385-4144-852a-457b111c58bf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.523572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d32f6-d385-4144-852a-457b111c58bf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.524053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d32f6-d385-4144-852a-457b111c58bf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.526352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 60/116] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:01.527647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d32f6-d385-4144-852a-457b111c58bf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.535419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:01.572460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164653e-56b1-4385-9afd-a4706ab2f904 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.610250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.610537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.610818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.612030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.613544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.615278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 57/117] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:01 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:01.616110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.652451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d32f6-d385-4144-852a-457b111c58bf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.694078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6d0b9-424b-4d0d-92c2-e2aec7a404f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.695493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d0b9-424b-4d0d-92c2-e2aec7a404f9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.695980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d0b9-424b-4d0d-92c2-e2aec7a404f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.696632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6d0b9-424b-4d0d-92c2-e2aec7a404f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.709892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6d0b9-424b-4d0d-92c2-e2aec7a404f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-20d6d0b9-424b-4d0d-92c2-e2aec7a404f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:01.717767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:01.720209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:01.726066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:01.726714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fe79b-d86b-46c3-8e56-9cb7099cddf5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.727133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 61/118] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:01.727133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96912178-f9c4-404d-b20b-3dfe79d7c7fc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.727133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96912178-f9c4-404d-b20b-3dfe79d7c7fc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.727133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96912178-f9c4-404d-b20b-3dfe79d7c7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.744128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-96912178-f9c4-404d-b20b-3dfe79d7c7fc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:01.745505 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-96912178-f9c4-404d-b20b-3dfe79d7c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:01.753967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:01.756998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.757696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.757991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.758340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.759082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.760547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.765543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:01.796665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 58/119] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:17:01 2022] POST /identity/v3/projects => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:01.796665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.836624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 62/120] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:01.839392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26374c41-7655-4f7a-a861-cf1b400ea262 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.839764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26374c41-7655-4f7a-a861-cf1b400ea262 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.840116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26374c41-7655-4f7a-a861-cf1b400ea262 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.845568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 63/121] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:01.846571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26374c41-7655-4f7a-a861-cf1b400ea262 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:01.928813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c013b9-3568-4d5b-b0e5-418c5e67c919 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.970050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba2c09-eaee-410a-a12a-53805dc6dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:01.970921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2c09-eaee-410a-a12a-53805dc6dd29 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.971215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2c09-eaee-410a-a12a-53805dc6dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.971496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2c09-eaee-410a-a12a-53805dc6dd29 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/b663cda428a149fa9a7a5c13fa738c8d/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:01.973833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba2c09-eaee-410a-a12a-53805dc6dd29 None admin] RBAC: Authorizing `identity:create_grant(project_id=1304cdebe410471bbff6abc40fce9b60, user_id=b663cda428a149fa9a7a5c13fa738c8d, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:01.975394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba2c09-eaee-410a-a12a-53805dc6dd29 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:01.986731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26374c41-7655-4f7a-a861-cf1b400ea262 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:01.993625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 59/122] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:01 2022] PUT /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/b663cda428a149fa9a7a5c13fa738c8d/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:01.995900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:01.995900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:01.995900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.021829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:02.023086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:02.048134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.049244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.049734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.050709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.051677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:02.052130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:02.054057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:02.068009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 64/123] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:02.069988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.076316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:02.082037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:02.120723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 60/124] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:02.121568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.212551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686a40b-719a-491b-b446-33836075ff9a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:02.255210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.258469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.258469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.258469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.279860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1434f-1c6d-43e7-b6c2-0b0d3e057560 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:02.281845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:02.298496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:02.299537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:02.300473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:02.308951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 65/125] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:02.311596 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.362023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.364211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.364553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.364835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.368046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:02.369556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:02.376597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 61/126] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:02.377491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.434422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:02.454035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:02.499029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5362f-693c-4cde-bde2-055be7670f99 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:02.539458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.540367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.540812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.541238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.542617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:02.543077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:02.544737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:02.553221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eaa60c-79b5-4316-8293-278beca1d882 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:02.555428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 62/127] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:02.561935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.675660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6168072-65ba-40b1-8f9b-4c33aab86a53 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:02.685379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:02.688784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.689684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.689977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.690266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.714296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.715437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.715867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.716288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.717683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:02.719381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:02.721662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 63/128] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:02.722749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.738348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:02.754509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:02.755621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 66/129] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:02.756604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.839509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adf9d0-73f9-4918-8937-7cdd488accb7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:02.878498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:02.879682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:02.880109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:02.880527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:02.882053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:02.883832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:02.885646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 64/130] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:02.886977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:02.949076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:03.005198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31489-95c6-4a3b-a9d3-be6cef30a208 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.016066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57bcbb-e87a-4b02-8b39-d5499595e1cb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.057555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:03.062433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.064430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.064886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.065330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.067133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:03.068501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.070781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.083412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.084534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.086430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.088946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/e50c121ca4544e98a148c3b6b1759f0d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.090090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 65/131] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:03.091587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:03.093324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] RBAC: Authorizing `identity:create_grant(project_id=1304cdebe410471bbff6abc40fce9b60, user_id=e50c121ca4544e98a148c3b6b1759f0d, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.095730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.130239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 67/132] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:02 2022] PUT /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/e50c121ca4544e98a148c3b6b1759f0d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:03.131456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:03.225702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a00c3a-d36e-45e2-bbfc-aa98d5a4bae5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.233753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:03.279167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.280097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.280382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.280661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] PATH_INFO: `/v3/projects/4146fddb1e44491196b3cb98804479c4` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.281949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] RBAC: Authorizing `identity:get_project(project_id=4146fddb1e44491196b3cb98804479c4)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.283445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.285217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 66/133] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:03 2022] GET /identity/v3/projects/4146fddb1e44491196b3cb98804479c4 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:03.291221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:03.304651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7a8d-04a3-4f83-865f-07802e38d7dc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.351797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.352780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.353071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.354456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.362335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.363861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.368210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 68/134] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:03 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:03.375912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:03.494905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:03.495165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301a48d-d054-4ea8-a8c2-3cb28df9c657 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.545069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.546036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.546323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.546602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] PATH_INFO: `/v3/projects/4146fddb1e44491196b3cb98804479c4/users/b663cda428a149fa9a7a5c13fa738c8d/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.549499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] RBAC: Authorizing `identity:create_grant(project_id=4146fddb1e44491196b3cb98804479c4, user_id=b663cda428a149fa9a7a5c13fa738c8d, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.551647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.570146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 67/135] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:03 2022] PUT /identity/v3/projects/4146fddb1e44491196b3cb98804479c4/users/b663cda428a149fa9a7a5c13fa738c8d/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:17:03.595818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b9d19-fc4d-4450-9998-022d8fadaab5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.649186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a264-52f0-4a40-9bd5-ae141b6b2e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.650024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a264-52f0-4a40-9bd5-ae141b6b2e87 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.650311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a264-52f0-4a40-9bd5-ae141b6b2e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.650594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a264-52f0-4a40-9bd5-ae141b6b2e87 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.651815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0a264-52f0-4a40-9bd5-ae141b6b2e87 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.653279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0a264-52f0-4a40-9bd5-ae141b6b2e87 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.655201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 69/136] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:03 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:03.660980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:03.710423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:03.798418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ff767-8f5c-4365-b050-01fe3cc318ba None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:03.839410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51edc28-6a38-4d35-99c2-0b1462600992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:03.840319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51edc28-6a38-4d35-99c2-0b1462600992 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:03.840827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51edc28-6a38-4d35-99c2-0b1462600992 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:03.840897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51edc28-6a38-4d35-99c2-0b1462600992 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/b663cda428a149fa9a7a5c13fa738c8d/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:03.847239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51edc28-6a38-4d35-99c2-0b1462600992 None admin] RBAC: Authorizing `identity:create_grant(project_id=1304cdebe410471bbff6abc40fce9b60, user_id=b663cda428a149fa9a7a5c13fa738c8d, role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:03.848721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51edc28-6a38-4d35-99c2-0b1462600992 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:03.864914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 68/137] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:03 2022] PUT /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/b663cda428a149fa9a7a5c13fa738c8d/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:04.195666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b0971-d9a3-4d80-aa10-11f24a67c464 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:04.196175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b0971-d9a3-4d80-aa10-11f24a67c464 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:04.196247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b0971-d9a3-4d80-aa10-11f24a67c464 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:04.197826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 70/138] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:04.204391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:04.204678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:04.204956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:04.223543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:04.224767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:04.331440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 69/139] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:04.370319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b0971-d9a3-4d80-aa10-11f24a67c464 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:04.519380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b0971-d9a3-4d80-aa10-11f24a67c464 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:04.551873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:04.572955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fa57e-79e7-4618-87f2-df4f8a3dddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:04.574081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fa57e-79e7-4618-87f2-df4f8a3dddb1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:04.574373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fa57e-79e7-4618-87f2-df4f8a3dddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:04.574652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fa57e-79e7-4618-87f2-df4f8a3dddb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:04.586393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fa57e-79e7-4618-87f2-df4f8a3dddb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1f3fa57e-79e7-4618-87f2-df4f8a3dddb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:04.595234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:04.596094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:04.596883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 71/140] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:04.603493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:04.633076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:04.656925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c38cec-15a4-42d3-bec8-bed80e962da0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:04.657254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c38cec-15a4-42d3-bec8-bed80e962da0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:04.657529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c38cec-15a4-42d3-bec8-bed80e962da0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:04.659077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 72/141] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:17:04.671283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:04.671458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:04.671728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:04.704098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:04.705322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:04.752242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc6411-6c81-4e7d-8ee5-6477e176ca31 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:04.824306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:04.824906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:04.825191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:04.825495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:04.826370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:04.827295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:04.828713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:04.834717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 73/142] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:04.843309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 70/143] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:04.849376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:04.876074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:05.010982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaab68b-bfe6-4e6a-a266-a6e2d8f85814 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.033128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:05.074898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:05.080115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d701a-6d70-4afb-8ba5-93956cd8e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.081020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d701a-6d70-4afb-8ba5-93956cd8e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.081306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d701a-6d70-4afb-8ba5-93956cd8e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.081601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d701a-6d70-4afb-8ba5-93956cd8e9b1 None admin] PATH_INFO: `/v3/users/00b23692a87e493a80a5261b70ab63c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.083903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d701a-6d70-4afb-8ba5-93956cd8e9b1 None admin] RBAC: Authorizing `identity:get_user(user_id=00b23692a87e493a80a5261b70ab63c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:05.085377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d701a-6d70-4afb-8ba5-93956cd8e9b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:05.090014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 74/144] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:04 2022] GET /identity/v3/users/00b23692a87e493a80a5261b70ab63c9 => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:05.091224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfcffd-20cb-413c-acb4-25554d55ee8f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.091515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfcffd-20cb-413c-acb4-25554d55ee8f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.091823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfcffd-20cb-413c-acb4-25554d55ee8f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.093432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 75/145] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:05.100766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfcffd-20cb-413c-acb4-25554d55ee8f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:05.111352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bfb49-a11d-4eaf-ab1a-d3f9dfa757e7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.157507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac1665-4eec-4d1d-86b7-977aaa5c8ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.159177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac1665-4eec-4d1d-86b7-977aaa5c8ceb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.159471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac1665-4eec-4d1d-86b7-977aaa5c8ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.159752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac1665-4eec-4d1d-86b7-977aaa5c8ceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.162757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac1665-4eec-4d1d-86b7-977aaa5c8ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:05.164541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac1665-4eec-4d1d-86b7-977aaa5c8ceb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:05.170521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 71/146] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:05.171961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.172251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.172532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.203281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:05.208390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:05.303502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfcffd-20cb-413c-acb4-25554d55ee8f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.351622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.352620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.352902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.353179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.355488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:05.356972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:05.359631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 72/147] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:05.360579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:05.364610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 76/148] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:05 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:05.371462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:05.425083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:05.508977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93610-a415-4a07-bb6d-5330320a1e27 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.569179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:05.576090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.577103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.577390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.577692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/00b23692a87e493a80a5261b70ab63c9/roles/6091ae42ec74461bbd8177eb121c6d63` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.580193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] RBAC: Authorizing `identity:create_grant(project_id=1304cdebe410471bbff6abc40fce9b60, user_id=00b23692a87e493a80a5261b70ab63c9, role_id=6091ae42ec74461bbd8177eb121c6d63)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:05.581670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:05.599374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04650c10-aac2-4c0e-919a-91dc34f9b7f7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.604021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 77/149] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:05 2022] PUT /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60/users/00b23692a87e493a80a5261b70ab63c9/roles/6091ae42ec74461bbd8177eb121c6d63 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:05.605710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:05.686225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.686923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.687221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.687484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.688212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:05.689716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:05.736497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 73/150] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:05 2022] POST /identity/v3/users => generated 283 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:05.743756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416388-71d4-450c-bcec-2177e38b939e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.753752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:05.790579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.791724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.792178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.792640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.811328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:05.819484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:05.820469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:05.821261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:05.823979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 78/151] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:05.830348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:05.950222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062e7d5-ad19-41cb-b937-ff3bf5cea27b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:05.992048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:05.993088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:05.993576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:05.993992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:05.994989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:05.995443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:05.997081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:06.007582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 74/152] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:06.013322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:06.134824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a47539-7738-4e1a-a2c9-791c99343a80 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:06.180199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:06.181320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:06.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:06.182205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:06.183554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:06.185141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:06.186990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 79/153] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:06 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:06.192451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:06.195569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:06.322432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefad03-b8ef-48e4-b4b4-678a3a87dc2d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:06.370725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4318-1b27-480c-9bb1-eced32fdb4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:06.371390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4318-1b27-480c-9bb1-eced32fdb4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:06.371678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4318-1b27-480c-9bb1-eced32fdb4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:06.371961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4318-1b27-480c-9bb1-eced32fdb4f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:06.465006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4318-1b27-480c-9bb1-eced32fdb4f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:06.466537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4318-1b27-480c-9bb1-eced32fdb4f2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:06.468640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:06.601205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 75/154] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:06.606553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:06.656211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:06.704660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df992055-68af-4b5a-aea6-5510403ea7fe None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:06.704981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df992055-68af-4b5a-aea6-5510403ea7fe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:06.705525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df992055-68af-4b5a-aea6-5510403ea7fe None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:06.708104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 76/155] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:06.715976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:06.716270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:06.716714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:06.755407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:06.756679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:06.767562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a5aea-fd26-4df3-a64a-90874ab38c54 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:06.824016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:06.824904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:06.825188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:06.825494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:06.832306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:06.833816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:06.835808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 80/156] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:06 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:06.841512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:06.886852 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 77/157] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:06.925450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:06.966111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:06.989295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264525e-b7c5-42ac-ba49-e64709348528 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:06.991160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:07.036081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:07.036749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.037038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.037314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.038197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=b663cda428a149fa9a7a5c13fa738c8d) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:07.038937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:07.040406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa8677-5989-4ed3-9743-5defd465c395 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:07.049016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 81/158] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:06 2022] GET /identity/v3/role_assignments?user.id=b663cda428a149fa9a7a5c13fa738c8d&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:07.050558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269c8d3-021b-460d-9cdf-e569ad4c7d85 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.050849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269c8d3-021b-460d-9cdf-e569ad4c7d85 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.051225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a269c8d3-021b-460d-9cdf-e569ad4c7d85 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.052758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 82/159] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:07.063554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.063887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.064113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.073188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9940277-3a2b-4c30-852b-f72ce5a773c2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:07.100984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:07.102309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:07.114620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08176b-653d-465b-9078-24ef7167042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:07.115439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08176b-653d-465b-9078-24ef7167042a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.115727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08176b-653d-465b-9078-24ef7167042a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.116002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08176b-653d-465b-9078-24ef7167042a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.129828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08176b-653d-465b-9078-24ef7167042a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1d08176b-653d-465b-9078-24ef7167042a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:07.139721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:07.140686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:07.141536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:07.142163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 78/160] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:07.145434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463edb1-cf0f-4486-9578-e8d6b626c0da None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.145434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463edb1-cf0f-4486-9578-e8d6b626c0da None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.145434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463edb1-cf0f-4486-9578-e8d6b626c0da None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.145923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 79/161] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:07.151807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.152136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.152355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.175303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:07.176523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:07.268582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 83/162] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:07.269513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:07.273278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 80/163] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:07.325163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:07.362910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:07.421710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:07.429879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba7265-7fe5-4470-8494-b5e8cd1e5de0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:07.483358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:07.484364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.484512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c38d7-5579-4f87-bacb-18af8fc4f928 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:07.485156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.485671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.486877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:07.488765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:07.490442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:07.502370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 84/164] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:07.503494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:07.528090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:07.528894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.529175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.529559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.548927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:07.556589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:07.557807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 81/165] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:07.559161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:07.644549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd13a6-ff31-4316-9940-21c1146b2006 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:07.684419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:07.685694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.686694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.687175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.699543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:07.709332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:07.710325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:07.711150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:07.713745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 85/166] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:07.714948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:07.745501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:07.752693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:07.785464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6dcd9-5759-4dc1-8168-0cf72c4ff4ce None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:07.854487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:07.855745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:07.861214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:07.861792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:07.863337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:07.865135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:07.873537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 82/167] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:07 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:07.874824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:07.926975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b44dc-3c29-40a0-973d-f1168ac96277 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.003795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376baf-af39-457f-90db-d439da69f44d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.018274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.019029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.019243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.019520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.020284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:08.020580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.022039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:08.042997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.044038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.044546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.045195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.046459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:08.046966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.048551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:08.049308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 86/168] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:08.050120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:08.059331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 83/169] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:08.060502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:08.076368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.163768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d36193-bb1c-439d-b7e1-e337f76cb660 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.198401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.199613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.200039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.200502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] PATH_INFO: `/v3/users/e50c121ca4544e98a148c3b6b1759f0d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.202194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] RBAC: Authorizing `identity:get_user(user_id=e50c121ca4544e98a148c3b6b1759f0d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.203818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:08.205980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 84/170] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:08 2022] GET /identity/v3/users/e50c121ca4544e98a148c3b6b1759f0d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:08.206991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:08.211546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.264921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13240b-eee8-451e-a297-8c6686b2405f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.290522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.310620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26a5a9-096a-4e55-b04c-ae77880fac3b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.341619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.342347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.342632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.342913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.346431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.347316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.347600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.347900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] PATH_INFO: `/v3/users/b663cda428a149fa9a7a5c13fa738c8d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.349101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] RBAC: Authorizing `identity:get_user(user_id=b663cda428a149fa9a7a5c13fa738c8d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.350655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:08.352306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 85/171] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:08 2022] GET /identity/v3/users/b663cda428a149fa9a7a5c13fa738c8d => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:08.353151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:08.455059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d4ecd-5434-456a-bc7d-9c8526571586 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.474917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.476423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:08.491203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.491907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.492196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.492474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.513435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.644492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.646052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:08.653481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.696466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 87/172] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:08.697153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:08.819549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29debe-fd4f-4717-ba7e-1dde7fefd759 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:08.878585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:08.879364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:08.879649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:08.879922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:08.905568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 86/173] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:08.906833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:08.963575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.992044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:08.992974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:08.994510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.087796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221943d-a012-4e7a-893e-7fb4cf7f71a5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:09.146823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:09.147737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.148016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.148292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] PATH_INFO: `/v3/projects/4146fddb1e44491196b3cb98804479c4` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.149532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] RBAC: Authorizing `identity:get_project(project_id=4146fddb1e44491196b3cb98804479c4)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:09.151022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.152921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 87/174] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:08 2022] GET /identity/v3/projects/4146fddb1e44491196b3cb98804479c4 => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:09.153523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:09.207101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 88/175] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:09.212757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:09.320449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:09.341167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f42c5-bbed-4f42-afd7-3c3c832a7747 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:09.379178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:09.379910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.380198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.380475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.381331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=4146fddb1e44491196b3cb98804479c4, user.id=b663cda428a149fa9a7a5c13fa738c8d) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:09.381679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:09.383197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.390275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 89/176] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:09 2022] GET /identity/v3/role_assignments?user.id=b663cda428a149fa9a7a5c13fa738c8d&scope.project.id=4146fddb1e44491196b3cb98804479c4&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:09.398545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:09.417171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cae7c4-839c-4fca-90f7-f9b07c367abc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:09.449908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:09.491096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:09.492000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.492276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.492548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.498462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:09.500213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.502138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 88/177] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:09 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:09.508180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:09.639854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb683004-6e5e-4267-a55d-f3139d8f4938 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:09.684028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55e494-943e-4c3d-afd4-8fc9d19441df None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:09.688728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35941cf5-b333-40a0-aa07-6c781b7e91aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:09.689682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35941cf5-b333-40a0-aa07-6c781b7e91aa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.689960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35941cf5-b333-40a0-aa07-6c781b7e91aa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.690230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35941cf5-b333-40a0-aa07-6c781b7e91aa None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.691430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35941cf5-b333-40a0-aa07-6c781b7e91aa None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:09.692880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35941cf5-b333-40a0-aa07-6c781b7e91aa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.695077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 90/178] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:09 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:09.696413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90744d-a0d9-4986-960c-6405e56ac982 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.696701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90744d-a0d9-4986-960c-6405e56ac982 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.696970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90744d-a0d9-4986-960c-6405e56ac982 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.698228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 91/179] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:09.702462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90744d-a0d9-4986-960c-6405e56ac982 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:09.721846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:09.737919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:09.738585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.738865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.739137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.739977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=e50c121ca4544e98a148c3b6b1759f0d) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:09.740284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:09.741760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7cc53-c6c1-427b-a74f-4d290b197843 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.748245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 89/180] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:09 2022] GET /identity/v3/role_assignments?user.id=e50c121ca4544e98a148c3b6b1759f0d&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:09.749280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.753734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.754084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.786510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:09.787734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:09.822827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90744d-a0d9-4986-960c-6405e56ac982 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:09.860987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:09.861692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:09.861972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:09.862303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:09.863148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] RBAC: Adding query filter params (role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=b663cda428a149fa9a7a5c13fa738c8d) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:09.863513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:09.864967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:09.871623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 92/181] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:09 2022] GET /identity/v3/role_assignments?user.id=b663cda428a149fa9a7a5c13fa738c8d&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:09.882148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:09.905585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 90/182] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:09.928768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.027295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.057042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76cdb-3bc5-42dc-9c64-de50fe9d9b88 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.096573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.097401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.097958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.098235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.110696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:10.118723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:10.119711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:10.120536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 93/183] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:10.126195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.246899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86a2f-d782-4967-98c6-c62025eca608 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.285614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.286331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.286613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.286881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.287637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:10.287939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:10.289736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5ad6-ef60-474b-99fb-9e5ad5ca7c3c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:10.298915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 91/184] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:10.304591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0b412-263f-4ba8-a7bd-8cdeaea1e295 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.311570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0b412-263f-4ba8-a7bd-8cdeaea1e295 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.311814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0b412-263f-4ba8-a7bd-8cdeaea1e295 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.313260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 92/185] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:10.319613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.319951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.320162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.352654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:10.353906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:10.397648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.442689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 93/186] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:10.444304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a272a-9107-43f7-8089-f45a579f190f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.469907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.488661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.490327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.491171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.491717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] PATH_INFO: `/v3/users/00b23692a87e493a80a5261b70ab63c9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.493239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] RBAC: Authorizing `identity:get_user(user_id=00b23692a87e493a80a5261b70ab63c9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:10.495790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:10.499635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 94/187] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/users/00b23692a87e493a80a5261b70ab63c9 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:10.505447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.515446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.588207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf116f-81f9-4b0b-9d73-a32107e8d74b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.623746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d87e23-ac8d-4fbd-a07e-bf55ec10fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.624653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d87e23-ac8d-4fbd-a07e-bf55ec10fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.624945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d87e23-ac8d-4fbd-a07e-bf55ec10fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.625228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d87e23-ac8d-4fbd-a07e-bf55ec10fa48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.627924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a9d9a-eca8-4ebb-a55f-5fae6961b097 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.635488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d87e23-ac8d-4fbd-a07e-bf55ec10fa48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a0d87e23-ac8d-4fbd-a07e-bf55ec10fa48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:10.643097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:10.644048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:10.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 94/188] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:10.650489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3e92-29e6-48e8-a78b-e55b86344e95 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.650775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3e92-29e6-48e8-a78b-e55b86344e95 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.651056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3e92-29e6-48e8-a78b-e55b86344e95 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.652591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 95/189] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:10.653221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3e92-29e6-48e8-a78b-e55b86344e95 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.670693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.671356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.671625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.671901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.688011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.745142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:10.746783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:10.750641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.774508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3e92-29e6-48e8-a78b-e55b86344e95 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.810875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.811860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.812142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.812416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.813169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:10.813500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:10.815157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb25b9e-2619-4d78-973d-33ceb14fe4bf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:10.823884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 96/190] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:10.824923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c649d9d-6ea9-4c2c-a540-816018ba16df None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.825203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c649d9d-6ea9-4c2c-a540-816018ba16df None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.825504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c649d9d-6ea9-4c2c-a540-816018ba16df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.843792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0c649d9d-6ea9-4c2c-a540-816018ba16df None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:10.844954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0c649d9d-6ea9-4c2c-a540-816018ba16df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:10.854904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 95/191] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:17:10.855754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.911300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 97/192] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:10.912800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926c2b8-4a93-41ad-8a21-20dbea80dd3f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.913147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926c2b8-4a93-41ad-8a21-20dbea80dd3f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.913786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926c2b8-4a93-41ad-8a21-20dbea80dd3f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:10.915199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 98/193] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:10.915780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926c2b8-4a93-41ad-8a21-20dbea80dd3f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:10.940093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.960419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08b6dc-83ad-4d75-9498-8935f12188ec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:10.962955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:10.995833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dfb3c-a313-43e2-91e7-1677efd165f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:10.996706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfb3c-a313-43e2-91e7-1677efd165f5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:10.996983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfb3c-a313-43e2-91e7-1677efd165f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:10.997255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfb3c-a313-43e2-91e7-1677efd165f5 None admin] PATH_INFO: `/v3/users/f2a81de40d6c49ababf5a3305ac27413` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.010570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dfb3c-a313-43e2-91e7-1677efd165f5 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a81de40d6c49ababf5a3305ac27413)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.011976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dfb3c-a313-43e2-91e7-1677efd165f5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.013753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 96/194] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/users/f2a81de40d6c49ababf5a3305ac27413 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.014927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.015210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.015483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.024065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926c2b8-4a93-41ad-8a21-20dbea80dd3f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.031432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:11.032632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:11.060210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.061489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.061780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.062057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.063254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.064715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.066607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 99/195] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:10 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.067219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.099388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 97/196] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:11.100222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.164631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.174901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50147d3b-2b0a-4499-ad6c-74d25294c198 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.192182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.201236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a86c43-bdca-45ad-aab5-e6c0f0fb96d8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.209270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.210111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.210387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.210655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.220810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:11.228715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:11.229824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:11.231342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 100/197] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:11.234511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.236749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.237402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.237825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.237989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.308950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.310431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.338560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd442-c37a-46c3-be20-d998367c20cc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.373042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.373978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.374262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.374540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.375378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] RBAC: Adding query filter params (role.id=6091ae42ec74461bbd8177eb121c6d63, scope.project.id=1304cdebe410471bbff6abc40fce9b60, user.id=00b23692a87e493a80a5261b70ab63c9) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:11.375692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.377165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.383220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 101/198] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/role_assignments?user.id=00b23692a87e493a80a5261b70ab63c9&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=6091ae42ec74461bbd8177eb121c6d63 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.383784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.401521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.409272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.424561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 98/199] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:11.424561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.514679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4683-1291-45b7-bf96-cfff357d1c26 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.538221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b3634-e814-43fc-860b-de7ea4a9e90b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.552582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.553480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.553797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.554067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.555551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.557033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.562231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 102/200] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.562231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.582800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.583882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.584275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.584555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.585309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:11.585699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.587104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.595892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 99/201] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.601197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.660786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.664834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.679417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc52a9-876e-4035-a9ee-7545af204232 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.709997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f923-cd0a-408c-a5ed-349f96f15e12 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.713360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.714262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.714546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.714826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.716010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.717660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.724361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 103/202] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.724361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.743567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.744237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.744510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.744794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.745537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.746986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.767583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 100/203] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:11 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:11.768373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.844163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325a40d-9939-4595-b810-7be58479f5a0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.884123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.885021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.885303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.885608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] PATH_INFO: `/v3/users/f2a81de40d6c49ababf5a3305ac27413` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.886842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a81de40d6c49ababf5a3305ac27413)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.888314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.890142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 104/204] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/users/f2a81de40d6c49ababf5a3305ac27413 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:11.891551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fcc11-7335-4cfd-be8b-2e167aae39eb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:11.895941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:11.905240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.918273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:11.933957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:11.934928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:11.935596 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:11.936162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:11.937448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=f2a81de40d6c49ababf5a3305ac27413) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:11.937930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:11.939449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:11.952086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 101/205] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/role_assignments?user.id=f2a81de40d6c49ababf5a3305ac27413&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:12.023406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c7a02-15b9-4848-94ca-4711d40dfdb0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:12.068913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:12.069670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:12.069971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:12.070228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:12.161512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:12.162889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:12.185931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:12.276934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 105/206] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:17:12.282667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:12.390421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a93aee-cf1f-4282-86c8-461f76dbb44d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:12.424769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16e8d0-a992-4699-be7d-a3b24c465753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:12.425578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16e8d0-a992-4699-be7d-a3b24c465753 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:12.425858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16e8d0-a992-4699-be7d-a3b24c465753 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:12.426130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16e8d0-a992-4699-be7d-a3b24c465753 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:12.427304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16e8d0-a992-4699-be7d-a3b24c465753 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:12.428762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16e8d0-a992-4699-be7d-a3b24c465753 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:12.430491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 102/207] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:12 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:12.436581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:12.544288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f37767-3911-4640-94a6-b660be2a9482 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:12.563688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:12.581762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:12.582770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:12.583274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:12.583793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:12.584871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] RBAC: Adding query filter params (role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=f2a81de40d6c49ababf5a3305ac27413) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:12.585379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:12.587048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68be7e-0395-4392-90ce-20e1682ab4fd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:12.593561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 106/208] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:12 2022] GET /identity/v3/role_assignments?user.id=f2a81de40d6c49ababf5a3305ac27413&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:14.211855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d1a4-24a6-43e8-97ea-2979b293d625 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.212383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d1a4-24a6-43e8-97ea-2979b293d625 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.212383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d1a4-24a6-43e8-97ea-2979b293d625 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.214082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 103/209] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:14.215054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b00c-2969-4d05-9b7b-2db45a5a4e86 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.215459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b00c-2969-4d05-9b7b-2db45a5a4e86 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.215613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b00c-2969-4d05-9b7b-2db45a5a4e86 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.217189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 107/210] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:14.222503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.222792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.223071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.225804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1418496-654b-4eb7-b0ea-c5c64b11b462 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.226090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1418496-654b-4eb7-b0ea-c5c64b11b462 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.226366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1418496-654b-4eb7-b0ea-c5c64b11b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.246309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:14.247604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d1418496-654b-4eb7-b0ea-c5c64b11b462 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:14.248267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:14.249119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d1418496-654b-4eb7-b0ea-c5c64b11b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:14.291378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:14.323688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 108/211] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:14.325009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01d304-7dee-4dfb-ba90-bc293ddc8afa None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.325300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01d304-7dee-4dfb-ba90-bc293ddc8afa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.325608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01d304-7dee-4dfb-ba90-bc293ddc8afa None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.327106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 109/212] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:14.332301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.332590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.332872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.334015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 104/213] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:14.350267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.354017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:14.355454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:14.431990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 110/214] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:14.433127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.474781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced104f-4f19-415e-b3ae-61013f417777 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:14.479385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:14.518976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:14.519938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.520570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.520868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.533640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:14.540750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5773b5-d3cd-456d-89fc-54a177269468 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:14.544918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:14.545883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:14.546692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:14.547322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 105/215] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:14.548103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.578603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:14.579409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.579752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.580164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.581205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:14.582712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:14.583575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:14.584624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 111/216] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:14.585120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.663637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fce50d-9aa2-4151-9de3-3f6f1aa0e7aa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:14.697545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:14.698347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.698630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.698905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.699198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad5288-b310-475a-8370-73518e987c1d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:14.709529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:14.712205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:14.718077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:14.719078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:14.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 106/217] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:14.720581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.744008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:14.744899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.745318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.745748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.746770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:14.747235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:14.748855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:14.762519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 112/218] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:14.763649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.820168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:14.829914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d5cf3-fb97-4e32-aa4d-2525a2b9296d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:14.861426 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd7fee-b709-4a64-8163-ecd7a00ab3e3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:14.864446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:14.865112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.865390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.865701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.866404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:14.867858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:14.888883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 107/219] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:14 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:14.890368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.895513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:14.896211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.896468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.896757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.897556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:14.897861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:14.899311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1f0f7-8e29-4943-9b6f-3ed8f788fcab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:14.909907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 113/220] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:14.911867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a9c19-a962-40ce-b587-3ffb496b7b66 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:14.912151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a9c19-a962-40ce-b587-3ffb496b7b66 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:14.912427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a9c19-a962-40ce-b587-3ffb496b7b66 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:14.917793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 114/221] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:14.920065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a9c19-a962-40ce-b587-3ffb496b7b66 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:14.961798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.009751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6336a-5ae4-4f43-b8fc-744a7d4c1d82 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.042337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a9c19-a962-40ce-b587-3ffb496b7b66 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.047907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89764e73-4cba-4710-bbab-590aaceacc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.048809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89764e73-4cba-4710-bbab-590aaceacc4b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.049091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89764e73-4cba-4710-bbab-590aaceacc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.049369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89764e73-4cba-4710-bbab-590aaceacc4b None admin] PATH_INFO: `/v3/users/9569ffe507a14325934895e3ea7a6a34` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.063998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89764e73-4cba-4710-bbab-590aaceacc4b None admin] RBAC: Authorizing `identity:get_user(user_id=9569ffe507a14325934895e3ea7a6a34)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.065528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89764e73-4cba-4710-bbab-590aaceacc4b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.071666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 108/222] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/users/9569ffe507a14325934895e3ea7a6a34 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:15.071666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41eee2-2293-472e-b838-0a409cac227c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.071666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41eee2-2293-472e-b838-0a409cac227c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.071666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41eee2-2293-472e-b838-0a409cac227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.080703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.087713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-be41eee2-2293-472e-b838-0a409cac227c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:15.088208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.089094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.089382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.089708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] PATH_INFO: `/v3/users/f2a81de40d6c49ababf5a3305ac27413` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.089967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-be41eee2-2293-472e-b838-0a409cac227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:15.090902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a81de40d6c49ababf5a3305ac27413)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.092775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.095169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 115/223] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:14 2022] GET /identity/v3/users/f2a81de40d6c49ababf5a3305ac27413 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:15.095957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.179139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 109/224] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:15.180277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41eee2-2293-472e-b838-0a409cac227c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.223550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5374ed9-7873-47f4-a99e-e6d33453e7f8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.226994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.275069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.275450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.275720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.275999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.314464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41eee2-2293-472e-b838-0a409cac227c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.350660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.351869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.352340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.352804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.354436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.356080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.356735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.358325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 110/225] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:15.358436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.359431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.363372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.474330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 116/226] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:15.475238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.477588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.483553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10113455-40c5-494e-bb56-2050886d8cb2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.519817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.520663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.520948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.521221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.535045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:15.549816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:15.550747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:15.551553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:15.555945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 111/227] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:15.556990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.596666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1ca29-e6dd-4317-8c56-ee547a65567e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.637865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.638407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.640714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.641005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.641322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/f2a81de40d6c49ababf5a3305ac27413/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.646199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] RBAC: Authorizing `identity:create_grant(project_id=4294a4df4bff4f8897d0a7b344ecbd36, user_id=f2a81de40d6c49ababf5a3305ac27413, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.647693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.663660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 117/228] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:15 2022] PUT /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/f2a81de40d6c49ababf5a3305ac27413/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:15.664475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.724392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a07b5-fbc0-4d45-b132-cc7609df0892 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.742291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:15.792983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59b13c-0d2c-41e2-8491-1f45a90d4ebb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:15.827398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.828691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.829169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.829767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.837519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.837519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.837519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 112/229] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:15.843979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:15.844717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:15.845007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:15.845283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:15.846132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:15.846444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:15.847917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:15.855590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:15.859254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 118/230] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:15.866514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:16.008611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3caef-c63b-41ee-9152-df866aba7799 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:16.022857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71014-81c1-4932-8b46-dadf92ecdb40 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:16.052598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:16.066133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:16.072323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:16.073066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:16.073503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:16.073848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:16.074762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=9569ffe507a14325934895e3ea7a6a34) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:16.075140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:16.076733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:16.082652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 113/231] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/role_assignments?user.id=9569ffe507a14325934895e3ea7a6a34&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:16.084112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:16.085021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:16.085308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:16.085611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] PATH_INFO: `/v3/users/f2a81de40d6c49ababf5a3305ac27413` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:16.090319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] RBAC: Authorizing `identity:get_user(user_id=f2a81de40d6c49ababf5a3305ac27413)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:16.091874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:16.094149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 119/232] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:15 2022] GET /identity/v3/users/f2a81de40d6c49ababf5a3305ac27413 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:16.099791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:16.250772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09c3fe-f070-4c48-b04a-878d43af0096 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:16.299393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000aa23-15c1-428b-8a4d-57ea0c2e23ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:16.300285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000aa23-15c1-428b-8a4d-57ea0c2e23ae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:16.300622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000aa23-15c1-428b-8a4d-57ea0c2e23ae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:16.300900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000aa23-15c1-428b-8a4d-57ea0c2e23ae None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:16.302320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000aa23-15c1-428b-8a4d-57ea0c2e23ae None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:16.303880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000aa23-15c1-428b-8a4d-57ea0c2e23ae None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:16.305944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 114/233] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:16 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:16.312504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:16.445049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6faad0-fd45-4544-88ec-b9cd1b937ecb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:16.489701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48fb25-e3b9-4b9d-825f-105760fbda6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:16.490627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48fb25-e3b9-4b9d-825f-105760fbda6f None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:16.490912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48fb25-e3b9-4b9d-825f-105760fbda6f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:16.491191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48fb25-e3b9-4b9d-825f-105760fbda6f None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/f2a81de40d6c49ababf5a3305ac27413/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:16.493536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48fb25-e3b9-4b9d-825f-105760fbda6f None admin] RBAC: Authorizing `identity:create_grant(project_id=4294a4df4bff4f8897d0a7b344ecbd36, user_id=f2a81de40d6c49ababf5a3305ac27413, role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:16.495100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48fb25-e3b9-4b9d-825f-105760fbda6f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:16.519401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 120/234] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:16 2022] PUT /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/f2a81de40d6c49ababf5a3305ac27413/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:17.578502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a4e5-e4d7-4df1-917e-ba647cfe83d4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:17.579501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a4e5-e4d7-4df1-917e-ba647cfe83d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:17.579984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356a4e5-e4d7-4df1-917e-ba647cfe83d4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:17.581908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 115/235] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:17.587749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:17.588601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:17.589094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:17.614405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:17.615968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:17.675885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:17.704317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 121/236] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:17.726543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917169-89bc-4331-9c04-ccc4f43387dd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:17.727005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:17.727371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917169-89bc-4331-9c04-ccc4f43387dd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:17.729167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917169-89bc-4331-9c04-ccc4f43387dd None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:17.731100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 116/237] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:17.736174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:17.736748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:17.737224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:17.765661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:17.767187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:17.834914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699196-8bec-430b-aa6e-38c43cb6acef None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:17.871526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:17.872597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:17.873078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:17.873537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:17.884913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:17.893708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:17.894631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:17.895431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:17.898007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 122/238] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:17.904154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:17.942915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:17.961034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:17.974549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 117/239] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:18.018022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:18.030716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae472ad6-714f-41b2-a0d9-6b4dd8574d28 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:18.101710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:18.102305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.102583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.102856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.103618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:18.103922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:18.105311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:18.118678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 123/240] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:18.126609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:18.226156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37604ca7-032b-4e6d-bd4b-e70c7ee8970e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:18.299847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:18.300645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.300923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.301193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.302509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:18.303961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:18.305611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 118/241] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:18.311663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:18.337107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21849a-4d7a-4b88-a24d-4ffe45543107 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:18.341250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:18.380164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd42cd9-2974-4f1c-9eaa-0a0ed36505e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:18.381455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd42cd9-2974-4f1c-9eaa-0a0ed36505e7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.381919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd42cd9-2974-4f1c-9eaa-0a0ed36505e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.382393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd42cd9-2974-4f1c-9eaa-0a0ed36505e7 None admin] PATH_INFO: `/v3/users/8fe833ed82024560b2c04be6a457c571` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.389678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:18.400201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd42cd9-2974-4f1c-9eaa-0a0ed36505e7 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe833ed82024560b2c04be6a457c571)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:18.402117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd42cd9-2974-4f1c-9eaa-0a0ed36505e7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:18.404503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 124/242] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:18 2022] GET /identity/v3/users/8fe833ed82024560b2c04be6a457c571 => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:18.406107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd1e1-47a8-4fe1-b7bc-76d988738e1c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.406601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd1e1-47a8-4fe1-b7bc-76d988738e1c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.407053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd1e1-47a8-4fe1-b7bc-76d988738e1c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.408483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 125/243] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:18.413862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd1e1-47a8-4fe1-b7bc-76d988738e1c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:18.434911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7380-523c-4a28-a9e5-42e9afb8e884 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:18.468977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2938d5-a99f-4543-999a-3ab86c093da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:18.469693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2938d5-a99f-4543-999a-3ab86c093da5 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.470135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2938d5-a99f-4543-999a-3ab86c093da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.470583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2938d5-a99f-4543-999a-3ab86c093da5 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.471468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2938d5-a99f-4543-999a-3ab86c093da5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:18.473155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2938d5-a99f-4543-999a-3ab86c093da5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:18.497399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 119/244] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:18 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:18.499758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.500840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.501354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.526069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:18.528091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:18.529052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd1e1-47a8-4fe1-b7bc-76d988738e1c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:18.583984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:18.585476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.586535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.587760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.615746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 120/245] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:18.653817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:18.656105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:18.751465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:18.755545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:18.766117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:18.851654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880e929-1418-4185-8e44-7272c23398b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:18.896868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:18.897509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:18.897797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:18.898080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:18.906318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 126/246] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:18.910201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:18.912394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:18.917806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:18.918796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:18.919687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 121/247] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:18.927186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.002795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.052370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268573d0-bb45-43d2-89db-98ac319248d9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.063185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dada2-db24-4415-ac5f-92ef6d2c47b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.064267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.102054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.102740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.103029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.103310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.104076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:19.104383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:19.106667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ed9eb-ca00-4d22-8a02-76b046d81a3f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:19.110469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.111627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.112087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.112537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.113958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:19.115868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:19.116917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 122/248] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:19.118018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 127/249] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:18 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:19.121444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15705-3764-4393-9adf-07a586e66e01 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.121444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15705-3764-4393-9adf-07a586e66e01 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.121444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de15705-3764-4393-9adf-07a586e66e01 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.121444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 123/250] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:19.124881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.126395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15705-3764-4393-9adf-07a586e66e01 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.292212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fc451-de03-4cee-954d-f6fa8b381c9e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.303890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de15705-3764-4393-9adf-07a586e66e01 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.324320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.331899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.332861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.333346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.333879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.335601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] RBAC: Adding query filter params (role.id=6091ae42ec74461bbd8177eb121c6d63, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=8fe833ed82024560b2c04be6a457c571) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:19.336375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:19.338467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd8b58-4fc8-40cc-b079-2aa31685f1d6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:19.344368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63633429-0605-458a-91a3-c72afe0c9cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.345294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63633429-0605-458a-91a3-c72afe0c9cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.345644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63633429-0605-458a-91a3-c72afe0c9cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.345914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63633429-0605-458a-91a3-c72afe0c9cf0 None admin] PATH_INFO: `/v3/users/f2a81de40d6c49ababf5a3305ac27413` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.346774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 128/251] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/role_assignments?user.id=8fe833ed82024560b2c04be6a457c571&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=6091ae42ec74461bbd8177eb121c6d63 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:19.347233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63633429-0605-458a-91a3-c72afe0c9cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=f2a81de40d6c49ababf5a3305ac27413)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:19.348485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09185a6-5597-435e-8f17-b80f2b46d93b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.348901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63633429-0605-458a-91a3-c72afe0c9cf0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:19.349217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09185a6-5597-435e-8f17-b80f2b46d93b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.349715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09185a6-5597-435e-8f17-b80f2b46d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.350834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 124/252] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/users/f2a81de40d6c49ababf5a3305ac27413 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:19.352218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3881e-5fa1-44dc-8874-54e00a46ef5a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.352500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3881e-5fa1-44dc-8874-54e00a46ef5a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.352770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3881e-5fa1-44dc-8874-54e00a46ef5a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.358656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 125/253] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:19.359855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3881e-5fa1-44dc-8874-54e00a46ef5a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.369605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e09185a6-5597-435e-8f17-b80f2b46d93b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:19.371118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e09185a6-5597-435e-8f17-b80f2b46d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:19.407440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.448186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 129/254] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:19.456144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.456976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.457498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.477183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3881e-5fa1-44dc-8874-54e00a46ef5a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.485041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:19.486654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:19.514825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.515500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.515785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.516057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.570601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 130/255] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:19.571925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.605353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:19.606895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:19.631838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.665391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.701628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4fb15-8416-47dd-bf84-861d01aaf417 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.742370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 126/256] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:19.743830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.744045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.744737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.745085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.745446 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.759435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:19.769745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:19.770643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:19.771676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 131/257] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:19.772542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.886485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bc506-da3d-4fc5-b058-0a2c6be26fbd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:19.901470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:19.941401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:19.943411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:19.946849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:19.947310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:19.948726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:19.950369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:19.955141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 132/258] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:19.958102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:19.975752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99be98-5916-46b4-be17-b653f67eec98 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.062227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.063212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.063328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.063524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.087753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:20.100333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29c9ad-12eb-4de0-852a-acfda9907a37 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.114014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:20.122181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:20.123442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 127/259] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:20.124644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:20.138854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.140143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.140602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.141121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.142140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:20.142636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:20.144233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:20.155579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 133/260] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:20.156740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:20.217793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:20.304066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc537f1-2a8b-4c22-899c-7aff21d8d2b1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.373023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0a51a-7acc-4f5f-9fbe-1be706ad95a9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.373951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.374846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.375294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.376676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.378215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:20.378692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:20.380388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:20.398219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 134/261] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:20.402019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:20.463015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.463689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.463973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.464249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.465096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=f2a81de40d6c49ababf5a3305ac27413) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:20.465425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:20.466914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:20.474352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 128/262] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:20 2022] GET /identity/v3/role_assignments?user.id=f2a81de40d6c49ababf5a3305ac27413&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:20.475172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:20.550209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34260-66f3-44eb-a562-dde71cbebc01 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.581803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:20.602583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:20.645129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffead55-de53-4651-970e-fcdb5992d5f2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.650131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.651043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.651330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.651615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] PATH_INFO: `/v3/users/9569ffe507a14325934895e3ea7a6a34` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.658302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] RBAC: Authorizing `identity:get_user(user_id=9569ffe507a14325934895e3ea7a6a34)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:20.659802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:20.661612 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 135/263] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:20 2022] GET /identity/v3/users/9569ffe507a14325934895e3ea7a6a34 => generated 297 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:20.667687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:20.682151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.684848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.685386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.685855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] PATH_INFO: `/v3/users/f2a81de40d6c49ababf5a3305ac27413` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.687499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] RBAC: Authorizing `identity:get_user(user_id=f2a81de40d6c49ababf5a3305ac27413)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:20.689308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:20.691958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 129/264] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:20 2022] GET /identity/v3/users/f2a81de40d6c49ababf5a3305ac27413 => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:20.697839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:20.874463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a4d64-7fbf-421d-aad7-603049c12fba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.922205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:20.929599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3d069-1dbe-4360-bb97-6b9da5d115c6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:20.934819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.935898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.937620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.938075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.977279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:20.978506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:20.981640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:20.982136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:20.984845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:20.986701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:20.988868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 136/265] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:20 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:20.996347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:21.076951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:21.078751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:21.080873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:21.191163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48aef7-b6f0-4b39-83f9-f2187734829e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:21.259799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:21.260757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.260971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.261243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/9569ffe507a14325934895e3ea7a6a34/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.265027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 130/266] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:21.266316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] RBAC: Authorizing `identity:create_grant(project_id=4294a4df4bff4f8897d0a7b344ecbd36, user_id=9569ffe507a14325934895e3ea7a6a34, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:21.267785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:21.270515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:21.293699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 137/267] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:20 2022] PUT /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/9569ffe507a14325934895e3ea7a6a34/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:21.311446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:21.407193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f514-0d4c-4495-9769-bf6f0179a9af None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:21.462444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c26eba-b038-4f15-b142-b350b454f4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:21.463258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c26eba-b038-4f15-b142-b350b454f4fc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.463535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c26eba-b038-4f15-b142-b350b454f4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.463803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c26eba-b038-4f15-b142-b350b454f4fc None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.465140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c26eba-b038-4f15-b142-b350b454f4fc None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:21.466670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c26eba-b038-4f15-b142-b350b454f4fc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:21.468399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 131/268] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:21 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:21.474391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:21.557296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5838-c52d-4b39-bcf4-947ff376e531 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.557971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5838-c52d-4b39-bcf4-947ff376e531 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.558434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f5838-c52d-4b39-bcf4-947ff376e531 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.560409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 132/269] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:21.566373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0d195-720c-4625-af6a-52ceb20905c7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.566997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0d195-720c-4625-af6a-52ceb20905c7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.567467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0d195-720c-4625-af6a-52ceb20905c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.588811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-07e0d195-720c-4625-af6a-52ceb20905c7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:21.591752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-07e0d195-720c-4625-af6a-52ceb20905c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:21.653748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e8218-331b-418a-8996-b00a9309b7dd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:21.666236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:21.708688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 133/270] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:21.710210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a5df-add2-4a82-a5c4-fa449d6817d1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.710500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a5df-add2-4a82-a5c4-fa449d6817d1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.710775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177a5df-add2-4a82-a5c4-fa449d6817d1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.712053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 134/271] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:21.717656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.718174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.718515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:21.718886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.719188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.719472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.719756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.720601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] RBAC: Adding query filter params (role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=f2a81de40d6c49ababf5a3305ac27413) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:21.720910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:21.725448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:21.729539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 138/272] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:21 2022] GET /identity/v3/role_assignments?user.id=f2a81de40d6c49ababf5a3305ac27413&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:21.736541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:21.746513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:21.747693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:21.775411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:21.826609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 135/273] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:21.850156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:21.853163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97cdd4-2352-41e8-988c-c9f2ed7a14a7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:21.889664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:21.890818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:21.891327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:21.891793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:21.893145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:21.894838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:21.896683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 139/274] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:21.902601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:21.941784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:21.973617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3bd9c-cb39-4bb5-8f55-666a6876c47f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.014926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d52cf-dc11-4cd7-a8c2-824190ee05be None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.016554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.017984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.019211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.019211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.036506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:22.048101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:22.049123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:22.049982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:22.051215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 136/275] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:22.057402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.060134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b8b44-6010-474d-be7f-0c37ee78a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.061316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8b44-6010-474d-be7f-0c37ee78a711 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.061832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8b44-6010-474d-be7f-0c37ee78a711 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.062313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8b44-6010-474d-be7f-0c37ee78a711 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.063224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b8b44-6010-474d-be7f-0c37ee78a711 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:22.064878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b8b44-6010-474d-be7f-0c37ee78a711 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:22.089740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:22.099050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 140/276] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:21 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:22.133940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da996b-49d6-42a3-8e9b-56419357c68c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.134252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da996b-49d6-42a3-8e9b-56419357c68c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.136318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da996b-49d6-42a3-8e9b-56419357c68c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.138077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 141/277] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:22.144176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.144512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.144791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.165448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:22.166671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:22.204158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a609765-20c3-41e4-b9d9-d1d21e9428fa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.245606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.246410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.246691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.246974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.247588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 142/278] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:22.249801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:22.250123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:22.251539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:22.261345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 137/279] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:22.268681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.281556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.299528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:22.385686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:22.408515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d892319-6866-4cfd-a322-bacf63a27428 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.414860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6227fe0-2723-4be9-8cd0-1478a6900c4f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.455433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.456246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.456531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.456810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.469235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.470151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.470436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.472425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] PATH_INFO: `/v3/users/e4a256fa8dd24914bf8e8fd3f9ad49de` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.474704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:22.483233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:22.484678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:22.485629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 138/280] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:22.490183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] RBAC: Authorizing `identity:get_user(user_id=e4a256fa8dd24914bf8e8fd3f9ad49de)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:22.491117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.491751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:22.494055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 143/281] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/users/e4a256fa8dd24914bf8e8fd3f9ad49de => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:22.500650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.614830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:22.635070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ccb85-cca3-40dc-b9f0-7726709f43c5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.639860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e855f7e-de43-48bc-a2d0-b0be4433fca7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.678398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.679051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.679336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.679613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.680683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, user.id=e4a256fa8dd24914bf8e8fd3f9ad49de, scope.system=all) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:22.680987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:22.681208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.681921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.682204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.682479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:22.682590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.683238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:22.683553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:22.685011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:22.689204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 144/282] 10.222.0.24 () {56 vars in 1308 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/role_assignments?user.id=e4a256fa8dd24914bf8e8fd3f9ad49de&scope.system=all&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:22.695892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 139/283] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:22.701863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.822208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648bbbd-8f27-4b81-bd30-7951457e6a5f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:22.865010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:22.865949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:22.866235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:22.866514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] PATH_INFO: `/v3/users/8fe833ed82024560b2c04be6a457c571` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:22.867815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] RBAC: Authorizing `identity:get_user(user_id=8fe833ed82024560b2c04be6a457c571)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:22.869202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:22.870897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 145/284] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/users/8fe833ed82024560b2c04be6a457c571 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:22.878722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:22.882351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:23.017278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804601c6-495b-48c8-b34e-47fdbe51212c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:23.071265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab50f4-bf48-4bf0-8d6b-b751ad6a4ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:23.072550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab50f4-bf48-4bf0-8d6b-b751ad6a4ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:23.073022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab50f4-bf48-4bf0-8d6b-b751ad6a4ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:23.073545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab50f4-bf48-4bf0-8d6b-b751ad6a4ea0 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:23.074924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab50f4-bf48-4bf0-8d6b-b751ad6a4ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:23.076848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab50f4-bf48-4bf0-8d6b-b751ad6a4ea0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:23.078799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 140/285] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:22 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:23.084487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:23.106453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:23.253556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633ad23-7d5e-495b-a042-ea96c3b47830 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:23.296131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca4456-7b69-407b-9479-e2a9fdbda00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:23.297032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca4456-7b69-407b-9479-e2a9fdbda00c None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:23.297322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca4456-7b69-407b-9479-e2a9fdbda00c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:23.297642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca4456-7b69-407b-9479-e2a9fdbda00c None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/8fe833ed82024560b2c04be6a457c571/roles/6091ae42ec74461bbd8177eb121c6d63` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:23.301351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca4456-7b69-407b-9479-e2a9fdbda00c None admin] RBAC: Authorizing `identity:create_grant(project_id=4294a4df4bff4f8897d0a7b344ecbd36, user_id=8fe833ed82024560b2c04be6a457c571, role_id=6091ae42ec74461bbd8177eb121c6d63)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:23.302920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca4456-7b69-407b-9479-e2a9fdbda00c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:23.325621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 146/286] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:17:23 2022] PUT /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/users/8fe833ed82024560b2c04be6a457c571/roles/6091ae42ec74461bbd8177eb121c6d63 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:24.187149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc473a-4020-43a8-8c80-d50e0d5b277d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:24.188174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc473a-4020-43a8-8c80-d50e0d5b277d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:24.188653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc473a-4020-43a8-8c80-d50e0d5b277d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:24.190320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 141/287] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:24.196043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:24.196505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:24.196583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:24.227525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:24.229624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:24.367487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 147/288] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:24.368545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:24.415070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc473a-4020-43a8-8c80-d50e0d5b277d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:24.516659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:24.559243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc473a-4020-43a8-8c80-d50e0d5b277d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:24.605892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:24.606695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:24.606975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:24.607341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:24.626300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:24.634026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:24.634917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:24.635737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:24.636874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 142/289] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:24.646544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:24.775264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5a85f-e62f-4443-8295-6d77d9a08f6f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:24.824792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:24.825495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:24.825801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:24.826247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:24.827037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:24.827345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:24.828739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:24.837906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 148/290] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:24.843339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:24.966273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008554f-88fe-4973-93b6-d889d2a23cdb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:24.995790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:25.002510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cfec7-bce3-48e5-9338-794cc9f909a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:25.003463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfec7-bce3-48e5-9338-794cc9f909a1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.003932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfec7-bce3-48e5-9338-794cc9f909a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.004213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfec7-bce3-48e5-9338-794cc9f909a1 None admin] PATH_INFO: `/v3/users/9569ffe507a14325934895e3ea7a6a34` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.005463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cfec7-bce3-48e5-9338-794cc9f909a1 None admin] RBAC: Authorizing `identity:get_user(user_id=9569ffe507a14325934895e3ea7a6a34)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:25.006942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cfec7-bce3-48e5-9338-794cc9f909a1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:25.008906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 143/291] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:24 2022] GET /identity/v3/users/9569ffe507a14325934895e3ea7a6a34 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:25.014412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.089856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:25.125746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a944-b6b9-4673-a52b-97bad8f73b30 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.126375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a944-b6b9-4673-a52b-97bad8f73b30 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.126892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a944-b6b9-4673-a52b-97bad8f73b30 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.128774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 144/292] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:25.133665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.134208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.137877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.141581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df9424-d144-4a96-9729-cdeba9e6b98a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:25.169345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:25.171164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:25.185857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:25.186519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.186803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.187078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.274199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:25.275905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:25.359958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 145/293] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:25.374863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:25.426882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 149/294] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:25.432586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.449079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.553877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aae04-cd02-4361-ab14-d28d8320cbf7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:25.560769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec3411-8993-4e60-bf6b-87d4ec36df7b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:25.582342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:25.591001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d289f4-80c4-4baf-bdae-7239db743a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:25.592082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d289f4-80c4-4baf-bdae-7239db743a7d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.592669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d289f4-80c4-4baf-bdae-7239db743a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.592946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d289f4-80c4-4baf-bdae-7239db743a7d None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.594152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d289f4-80c4-4baf-bdae-7239db743a7d None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:25.596030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d289f4-80c4-4baf-bdae-7239db743a7d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:25.597164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a6a2d-7a21-4ae6-8f20-b7cedc71afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:25.602557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a6a2d-7a21-4ae6-8f20-b7cedc71afe7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.602700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 146/295] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:25.602792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a6a2d-7a21-4ae6-8f20-b7cedc71afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.602792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a6a2d-7a21-4ae6-8f20-b7cedc71afe7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.602940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fee72-eb12-4125-963f-2c739d418308 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.602940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fee72-eb12-4125-963f-2c739d418308 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.602940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fee72-eb12-4125-963f-2c739d418308 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.602940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 147/296] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:25.606336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e74d73-a302-40c5-aae2-7aec7f735d0c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.606623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e74d73-a302-40c5-aae2-7aec7f735d0c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.606893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e74d73-a302-40c5-aae2-7aec7f735d0c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.608387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 148/297] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:25.608938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e74d73-a302-40c5-aae2-7aec7f735d0c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.619352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0a6a2d-7a21-4ae6-8f20-b7cedc71afe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7d0a6a2d-7a21-4ae6-8f20-b7cedc71afe7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:25.627790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:25.628699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 150/298] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:25.634684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0cf48-d863-4bcd-8ed6-ca3de12411e3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.635423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0cf48-d863-4bcd-8ed6-ca3de12411e3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.635423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0cf48-d863-4bcd-8ed6-ca3de12411e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.649046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5ba0cf48-d863-4bcd-8ed6-ca3de12411e3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:25.650347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5ba0cf48-d863-4bcd-8ed6-ca3de12411e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:25.668236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:25.722611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e74d73-a302-40c5-aae2-7aec7f735d0c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:25.723324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 151/299] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:25.727153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.727442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.727722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.753153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:25.754523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:25.761957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:25.763030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.763368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.763653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.764867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=9569ffe507a14325934895e3ea7a6a34) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:25.765183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:25.766981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:25.773738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 149/300] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/role_assignments?user.id=9569ffe507a14325934895e3ea7a6a34&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:25.774376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.852967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 152/301] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:25.854293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.858099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:25.912911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8389e22-4517-449a-98c6-4bfe394c8fa6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:25.952337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:25.953035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:25.953321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:25.953630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:25.954683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:25.954996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:25.957170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:25.969060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 150/302] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:25.969978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:25.983738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.007204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5904e40-fbb3-407f-842b-2a8eed7395d5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.062313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.063969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.064595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.065131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.066699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.069106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.070963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 153/303] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:26.071912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.098894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31636a6-c469-4eeb-8703-dc2f0a5a51ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.120861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.133974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.134779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.135063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.135339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.146099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:26.153717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:26.154898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:26.155725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 151/304] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:26.156513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.192154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55c3a1-2f6d-4870-a6ba-9fb4ea8027da None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.226378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.227273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.227555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.227828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] PATH_INFO: `/v3/users/ee9254eaa3a64b18bd81fa0023b611ad` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.246471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] RBAC: Authorizing `identity:get_user(user_id=ee9254eaa3a64b18bd81fa0023b611ad)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.249221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.251006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 154/305] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/users/ee9254eaa3a64b18bd81fa0023b611ad => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:26.251773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.263317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.275438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec650d93-7d2c-4996-8ecf-231d3bde3045 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.309067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70608ce0-d600-46c5-ad0e-5e31f20a9c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.309760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608ce0-d600-46c5-ad0e-5e31f20a9c6e None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.310039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608ce0-d600-46c5-ad0e-5e31f20a9c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.310426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70608ce0-d600-46c5-ad0e-5e31f20a9c6e None admin] PATH_INFO: `/v3/groups` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.311111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608ce0-d600-46c5-ad0e-5e31f20a9c6e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.312564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70608ce0-d600-46c5-ad0e-5e31f20a9c6e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.327721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 152/306] 10.222.0.24 () {60 vars in 1134 bytes} [Thu May 19 13:17:26 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:26.329322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06c88a-b09f-4f87-8ce2-05221dade3c5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.329639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06c88a-b09f-4f87-8ce2-05221dade3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.329922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06c88a-b09f-4f87-8ce2-05221dade3c5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.331185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 153/307] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:26.333605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06c88a-b09f-4f87-8ce2-05221dade3c5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.375353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686d58f-e1d2-4b9c-aa05-f13efc16f8bb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.383206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.415363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.416052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.416340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.416616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.417399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:26.417740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.419162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271bb7c-eaec-4ce9-b3f6-dd019df8817c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.428603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 155/308] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:26.430069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.430362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.430640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.448027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:26.449159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:26.456943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06c88a-b09f-4f87-8ce2-05221dade3c5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.496250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.496963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.497253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.497569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.499142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] RBAC: Adding query filter params (role.id=6091ae42ec74461bbd8177eb121c6d63, user.id=ee9254eaa3a64b18bd81fa0023b611ad, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:26.499464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.500982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.507136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 154/309] 10.222.0.24 () {56 vars in 1308 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/role_assignments?user.id=ee9254eaa3a64b18bd81fa0023b611ad&scope.system=all&role.id=6091ae42ec74461bbd8177eb121c6d63 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:26.508026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.539865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.546127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 156/310] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:26.570482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.645461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465fd81-c50f-48c8-b47b-9d4431f4d8b2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.654603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.696777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.698706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.698815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.699001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] PATH_INFO: `/v3/users/e4a256fa8dd24914bf8e8fd3f9ad49de` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.700600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] RBAC: Authorizing `identity:get_user(user_id=e4a256fa8dd24914bf8e8fd3f9ad49de)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.702429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.704851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 155/311] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/users/e4a256fa8dd24914bf8e8fd3f9ad49de => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:26.710370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.715326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ae46b-1643-49b0-a3c3-da2a8b2de18c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.751555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.752760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.753260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.753837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.764567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:26.772721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:26.773677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:26.774503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:26.776964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 157/312] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:26.782658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:26.834948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00966a-2341-4708-bc99-afa18125153a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.849775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:26.872281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.873347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.873609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.873800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] PATH_INFO: `/v3/system/users/e4a256fa8dd24914bf8e8fd3f9ad49de/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.875295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e4a256fa8dd24914bf8e8fd3f9ad49de, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.876743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.889911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 156/313] 10.222.0.24 () {58 vars in 1334 bytes} [Thu May 19 13:17:26 2022] PUT /identity/v3/system/users/e4a256fa8dd24914bf8e8fd3f9ad49de/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:26.898684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab150d3-4a84-4500-af78-57f5c7f5116f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:26.937584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:26.938652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:26.939144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:26.939605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:26.940595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:26.941001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:26.942693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:26.952816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 158/314] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:26.958166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:27.008202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:27.081767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff0bb-b2d2-4a80-a4b4-c2daa51fbb4a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:27.119738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:27.120627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:27.120908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:27.121183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] PATH_INFO: `/v3/users/8fe833ed82024560b2c04be6a457c571` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:27.122407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] RBAC: Authorizing `identity:get_user(user_id=8fe833ed82024560b2c04be6a457c571)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:27.124158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:27.126145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 157/315] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:26 2022] GET /identity/v3/users/8fe833ed82024560b2c04be6a457c571 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:27.131654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:27.235449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc50276-7aa8-4ae7-97be-f5c55ab4c24d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:27.269157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:27.271433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:27.272159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:27.272679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:27.272962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:27.346082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:27.347554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:27.447294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 159/316] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:27.453142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:27.553045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3f585-896c-4518-b35f-bd875de8878d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:27.563101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:27.589577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b65eac-4aae-4c66-ad3b-3f0ce2e602a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:27.590737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65eac-4aae-4c66-ad3b-3f0ce2e602a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:27.591189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65eac-4aae-4c66-ad3b-3f0ce2e602a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:27.591689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65eac-4aae-4c66-ad3b-3f0ce2e602a2 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:27.593050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b65eac-4aae-4c66-ad3b-3f0ce2e602a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:27.594702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b65eac-4aae-4c66-ad3b-3f0ce2e602a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:27.596572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 158/317] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:27 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:27.602573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:27.634139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:27.715056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce5f4-2440-466e-b66a-88ebd02053e1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:27.749803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:27.750761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:27.751304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:27.751745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:27.752807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] RBAC: Adding query filter params (role.id=6091ae42ec74461bbd8177eb121c6d63, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36, user.id=8fe833ed82024560b2c04be6a457c571) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:27.753311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:27.754929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d780577-80db-4135-ae50-c31b5025b0ad None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:27.761590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 160/318] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:27 2022] GET /identity/v3/role_assignments?user.id=8fe833ed82024560b2c04be6a457c571&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=6091ae42ec74461bbd8177eb121c6d63 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:28.670136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699db7d-7a66-4078-af93-7ba19e62c581 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:28.670663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699db7d-7a66-4078-af93-7ba19e62c581 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:28.670930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699db7d-7a66-4078-af93-7ba19e62c581 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:28.672169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 159/319] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:28.677636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:28.678051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:28.678219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:28.696002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:28.697331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:28.738879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128573-e902-4801-90c9-59e8d21ce2ce None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:28.739132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128573-e902-4801-90c9-59e8d21ce2ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:28.739673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128573-e902-4801-90c9-59e8d21ce2ce None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:28.741065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 160/320] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:17:28.746323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:28.746610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:28.747111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:28.764627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:28.766216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:28.766399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 161/321] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:28.788631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:28.806667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:28.840897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 161/322] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:28.863108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:28.895988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516afff9-98f2-44dc-b4e4-a1292c3097ea None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:28.937452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:28.938242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:28.938521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:28.938793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:28.939948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:28.941401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:28.943199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 162/323] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:28.948486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:28.966647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fdafb-2a34-4951-9826-cd480e19d4fa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:28.970444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:29.007062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.008003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.008283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.008674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.021132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:29.029756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:29.030927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:29.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 162/324] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:29.036003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:29.037506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:29.053965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c7438-51ee-4dc7-9af5-9c54fd078607 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.088322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc21079-5290-4e12-8f69-f14af444a0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.089013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc21079-5290-4e12-8f69-f14af444a0c9 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.089291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc21079-5290-4e12-8f69-f14af444a0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.089563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc21079-5290-4e12-8f69-f14af444a0c9 None admin] PATH_INFO: `/v3/groups` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.090283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc21079-5290-4e12-8f69-f14af444a0c9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:29.091748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc21079-5290-4e12-8f69-f14af444a0c9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:29.117436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 163/325] 10.222.0.24 () {60 vars in 1134 bytes} [Thu May 19 13:17:28 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:17:29.155654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd02d32-e295-485d-9ed4-870de5b7cef5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.192803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.193390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.193707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.193983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.194742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:29.195044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:29.196438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:29.206160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 163/326] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:29.211526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d6a52-30a4-478c-a1f3-ecc2d4a9118c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.211712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:29.211825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d6a52-30a4-478c-a1f3-ecc2d4a9118c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.212179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d6a52-30a4-478c-a1f3-ecc2d4a9118c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.213713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 164/327] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:29.220067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.220354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.220631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.240234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:29.241462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:29.244023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:29.324951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 165/328] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:29.331550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49303d6-ba52-4463-b7b1-58b04d30333c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.348944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:29.369176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.370122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.370405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.370683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] PATH_INFO: `/v3/users/ee9254eaa3a64b18bd81fa0023b611ad` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.371877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=ee9254eaa3a64b18bd81fa0023b611ad)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:29.373587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:29.375247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 164/329] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:29 2022] GET /identity/v3/users/ee9254eaa3a64b18bd81fa0023b611ad => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:29.380799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:29.436996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:29.470958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e94405-5a61-4a9d-87dd-d50c19df033e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.500542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:29.509063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1c36-f463-4ff3-b2c4-a26a90b02fa4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.513089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.513982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.514261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.514539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.530283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:29.538053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:29.539484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:29.540348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 166/330] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:29.545746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:29.560701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.561767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.562121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.562467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] PATH_INFO: `/v3/system/users/ee9254eaa3a64b18bd81fa0023b611ad/roles/6091ae42ec74461bbd8177eb121c6d63` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.564193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ee9254eaa3a64b18bd81fa0023b611ad, role_id=6091ae42ec74461bbd8177eb121c6d63)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:29.565711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:29.575818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 165/331] 10.222.0.24 () {58 vars in 1334 bytes} [Thu May 19 13:17:29 2022] PUT /identity/v3/system/users/ee9254eaa3a64b18bd81fa0023b611ad/roles/6091ae42ec74461bbd8177eb121c6d63 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:17:29.665751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865ee4-af8d-416e-9087-6591b4c5314d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.709351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.710100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.710345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.710656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.711840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:29.712360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:29.714259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:29.715890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:29.726346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 167/332] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:29.732321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:29.845767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c29de-d669-4c38-b1b2-adbe2dacbb57 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:29.880106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89cb6e-7c73-4aff-99c4-99720357318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:29.881250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89cb6e-7c73-4aff-99c4-99720357318f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:29.881699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89cb6e-7c73-4aff-99c4-99720357318f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:29.882134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89cb6e-7c73-4aff-99c4-99720357318f None admin] PATH_INFO: `/v3/users/e4a256fa8dd24914bf8e8fd3f9ad49de` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:29.883479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89cb6e-7c73-4aff-99c4-99720357318f None admin] RBAC: Authorizing `identity:get_user(user_id=e4a256fa8dd24914bf8e8fd3f9ad49de)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:29.885055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89cb6e-7c73-4aff-99c4-99720357318f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:29.887207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 166/333] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:29 2022] GET /identity/v3/users/e4a256fa8dd24914bf8e8fd3f9ad49de => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:29.892769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:30.015055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b67a9-88d2-4b5a-859a-ae2ac13890af None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:30.054340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:30.055243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:30.055683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:30.056107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:30.057168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] RBAC: Adding query filter params (role.id=cc93bbcc688e4a08879b0b936b6018ea, user.id=e4a256fa8dd24914bf8e8fd3f9ad49de, scope.system=all) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:30.057647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:30.059226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a854d0d-58f5-424f-998f-31bbeb581f56 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:30.074188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 168/334] 10.222.0.24 () {56 vars in 1308 bytes} [Thu May 19 13:17:29 2022] GET /identity/v3/role_assignments?user.id=e4a256fa8dd24914bf8e8fd3f9ad49de&scope.system=all&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:31.673952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c8c1e-dd61-498d-bb02-4328dbd5ddc6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:31.674976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c8c1e-dd61-498d-bb02-4328dbd5ddc6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:31.675474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c8c1e-dd61-498d-bb02-4328dbd5ddc6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:31.677159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 167/335] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:31.683283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:31.683824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:31.683824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:31.702872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:31.704103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:31.760929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:31.780219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 169/336] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:31.804954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c8c1e-dd61-498d-bb02-4328dbd5ddc6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:31.847544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:31.940504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c8c1e-dd61-498d-bb02-4328dbd5ddc6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:31.980805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3f181-96d7-4ec1-974a-282fb36d9a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:31.981626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f181-96d7-4ec1-974a-282fb36d9a11 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:31.981935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f181-96d7-4ec1-974a-282fb36d9a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:31.982305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f181-96d7-4ec1-974a-282fb36d9a11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:31.994971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3f181-96d7-4ec1-974a-282fb36d9a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-25f3f181-96d7-4ec1-974a-282fb36d9a11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:32.003064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:32.004039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:32.005520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 168/337] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:32.010121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:32.058516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493607d2-6f89-4e3a-872e-9ff132c14bab None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.058730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493607d2-6f89-4e3a-872e-9ff132c14bab None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.059001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493607d2-6f89-4e3a-872e-9ff132c14bab None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.061052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 169/338] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:32.068342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.068625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.068895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.098803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:32.099995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:32.190918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd4827-bbe1-4e09-9b40-21ddf0cebfc5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:32.207180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 170/339] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:32.267922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:32.278692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:32.279674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.280193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.280627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.281642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:32.282114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:32.283730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:32.299748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 170/340] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:32.305203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:32.314326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:32.389782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:32.434307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d3f71-e960-4d6b-bde8-82243cbb5f4a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:32.487501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:32.488333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.488613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.488887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.493097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61a294-3552-4628-98fd-13267c483033 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:32.506851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:32.523745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:32.524722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:32.526031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 171/341] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:32.531267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:32.551127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:32.551796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.552076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.552347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.652097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:32.652442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:32.673838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:32.714574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef8651-4990-4fd1-9f2f-76b53b287b22 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:32.722889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:32.794456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:32.795675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.796095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.796473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.797253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:32.797652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:32.799687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:32.804423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 171/342] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:17:32.809937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:32.816061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 172/343] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:32.823064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:32.937786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089c714-2a86-4858-b2ce-db8430a39b79 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:32.975857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:32.976743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:32.977022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:32.977361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:32.978745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:32.980262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:32.983715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 172/344] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:32.991579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:33.006936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cad1ec-89bc-431e-b0a7-8229a85a87be None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:33.012737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:33.108462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36752cf-77a3-4f6e-86c8-6149b3951d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:33.109057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36752cf-77a3-4f6e-86c8-6149b3951d98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.109342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36752cf-77a3-4f6e-86c8-6149b3951d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.109645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36752cf-77a3-4f6e-86c8-6149b3951d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.131948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:33.134491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341ae78-70e7-4a62-8902-86f35dfb8d70 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:33.174594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:33.175465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.175746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.176022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.183020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:33.184490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:33.188969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 173/345] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:33.194867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:33.242583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36752cf-77a3-4f6e-86c8-6149b3951d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:33.245323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36752cf-77a3-4f6e-86c8-6149b3951d98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:33.303715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:33.341856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c4115-ddb2-4f26-8aa0-26adbc7096a4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:33.399528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:33.400645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.401179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.402023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.402977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=1304cdebe410471bbff6abc40fce9b60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:33.405028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:33.405808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 173/346] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:17:33.406847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cc1aa-1585-4fd2-9748-eff064476f1d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:33.407559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746c259-e0c0-4009-ac8a-e0d03c56252a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.407987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746c259-e0c0-4009-ac8a-e0d03c56252a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.408409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746c259-e0c0-4009-ac8a-e0d03c56252a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.414232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 174/347] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:33.415599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da551a-3e59-437f-9593-1973a7a381ac None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.415599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da551a-3e59-437f-9593-1973a7a381ac None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.415599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da551a-3e59-437f-9593-1973a7a381ac None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.418121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 175/348] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:33.418626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da551a-3e59-437f-9593-1973a7a381ac None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:33.421528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 174/349] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:33 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:33.422904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342f11-2cbc-4d44-b790-b09b4619a176 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.423188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342f11-2cbc-4d44-b790-b09b4619a176 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.423459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342f11-2cbc-4d44-b790-b09b4619a176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.459183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-99342f11-2cbc-4d44-b790-b09b4619a176 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:33.460858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-99342f11-2cbc-4d44-b790-b09b4619a176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:33.486203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:33.581763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da551a-3e59-437f-9593-1973a7a381ac None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:33.631313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 175/350] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:33.636164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b119c98-a5ea-4056-bdf4-2604cf05715b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.636734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b119c98-a5ea-4056-bdf4-2604cf05715b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.637221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b119c98-a5ea-4056-bdf4-2604cf05715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.642693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:33.643592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.643933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.644220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.646284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:33.648684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:33.661889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 176/351] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:33 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:33.664047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:33.669643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9b119c98-a5ea-4056-bdf4-2604cf05715b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:33.671192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9b119c98-a5ea-4056-bdf4-2604cf05715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:33.778053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 176/352] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:33.779832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb64ab-e2cc-4826-aa7d-8f9c5c7c3cc8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.780158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb64ab-e2cc-4826-aa7d-8f9c5c7c3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.780441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb64ab-e2cc-4826-aa7d-8f9c5c7c3cc8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.786315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 177/353] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:33.787261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb64ab-e2cc-4826-aa7d-8f9c5c7c3cc8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:33.797888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:33.802973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000824e-babf-49e1-b834-7498cfe706d8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:33.810834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:33.840328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de199c72-87cf-4e18-ad6f-3e58c14b5aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:33.841194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de199c72-87cf-4e18-ad6f-3e58c14b5aab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.841624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de199c72-87cf-4e18-ad6f-3e58c14b5aab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.841999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de199c72-87cf-4e18-ad6f-3e58c14b5aab None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.843369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de199c72-87cf-4e18-ad6f-3e58c14b5aab None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:33.844821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de199c72-87cf-4e18-ad6f-3e58c14b5aab None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:33.846915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 177/354] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:33 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:33.848163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.848507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.848778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.880608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:33.881960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:33.937003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb64ab-e2cc-4826-aa7d-8f9c5c7c3cc8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:33.960996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 178/355] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:33.961903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:33.981054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:33.982024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:33.982473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:33.982802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:33.997821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:34.008503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:34.009473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:34.010745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 178/356] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:34.014431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.087732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123e937-78bb-4672-953b-5c41daa680b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.090153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:34.093736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:34.128814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.129796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.130283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.130657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.137233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95815fe2-724e-4e3f-bbff-26c29ec732d5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.143410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:34.153134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:34.154364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 13:17:34.155409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:34.157129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 179/357] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:34.158093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.186387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.187169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.187397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.187672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.188639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=1304cdebe410471bbff6abc40fce9b60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:34.188957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:34.190494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:34.199320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 179/358] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:34.201650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.321179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc19da1-09d7-48c0-aa5d-533558ce19ad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.371805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.372999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.373449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.373872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.388152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:34.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:34.399165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:34.399994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:34.403582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 180/359] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:34.405013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.447389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:34.449015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07a950-15d5-4e29-9a20-7e7bd95a6038 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.491454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:34.512534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.513721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.514020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.514312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.515071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:34.515399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:34.516777 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:34.532267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 180/360] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:34.533117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.560909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a72be-b7e2-4a23-b39d-eb89505d7e4a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.630711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.631391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.631672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.631942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.632708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:34.633014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:34.634430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:34.652658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 181/361] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:34.653534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.675786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb0562-600b-4bcb-807d-bad33c925f07 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.722315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.723449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.723863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.724292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.725182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:34.725765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:34.727843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:34.737340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 181/362] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:34.742559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:34.804039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:34.806223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:34.844518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1066ac1-89d9-4320-aedb-3c02217245eb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.875987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f194f5d-bcf4-4fc5-a978-010628e67e96 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:34.928028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.928686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.928965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.931076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.933337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:34.934268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:34.934547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:34.934821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] PATH_INFO: `/v3/users/ee9254eaa3a64b18bd81fa0023b611ad` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:34.938736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] RBAC: Authorizing `identity:get_user(user_id=ee9254eaa3a64b18bd81fa0023b611ad)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:34.940211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:34.941916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 182/363] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/users/ee9254eaa3a64b18bd81fa0023b611ad => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:34.942674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.033967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.035459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.082423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481a646-398f-4510-8cff-faea498a44bb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:35.101922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:35.122508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e940f57d-dd56-45a0-a665-e1a099d5d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:35.123539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940f57d-dd56-45a0-a665-e1a099d5d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.124044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940f57d-dd56-45a0-a665-e1a099d5d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.124514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e940f57d-dd56-45a0-a665-e1a099d5d8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.189026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 182/364] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:35.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.212137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940f57d-dd56-45a0-a665-e1a099d5d8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.214450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e940f57d-dd56-45a0-a665-e1a099d5d8a2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.221845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:35.308792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc58e5-490c-40ed-ae80-4221626a3491 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:35.339417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 183/365] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:35.341658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f76a97-a62b-4d9f-8a20-1cfc4b3349ff None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.342268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f76a97-a62b-4d9f-8a20-1cfc4b3349ff None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.342689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f76a97-a62b-4d9f-8a20-1cfc4b3349ff None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.344756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 184/366] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:35.346813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f76a97-a62b-4d9f-8a20-1cfc4b3349ff None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.348556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:35.349706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.350182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.350744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.351785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] RBAC: Adding query filter params (role.id=6091ae42ec74461bbd8177eb121c6d63, user.id=ee9254eaa3a64b18bd81fa0023b611ad, scope.system=all) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:35.352294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.354062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.367675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:35.368064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 183/367] 10.222.0.24 () {56 vars in 1308 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/role_assignments?user.id=ee9254eaa3a64b18bd81fa0023b611ad&scope.system=all&role.id=6091ae42ec74461bbd8177eb121c6d63 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:35.369957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.425789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:35.499682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f76a97-a62b-4d9f-8a20-1cfc4b3349ff None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:35.520385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d05f6-ef97-4a8e-a8a1-9b865441e696 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:35.567553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa3eb2-4dfa-4452-ab57-6c9bb8cb79e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:35.568351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa3eb2-4dfa-4452-ab57-6c9bb8cb79e0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.568633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa3eb2-4dfa-4452-ab57-6c9bb8cb79e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.569738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa3eb2-4dfa-4452-ab57-6c9bb8cb79e0 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.571128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa3eb2-4dfa-4452-ab57-6c9bb8cb79e0 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.572622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa3eb2-4dfa-4452-ab57-6c9bb8cb79e0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.581077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:35.582002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.582287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.582561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.583549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 185/368] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:35.584909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.585194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.587538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.587978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.591000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.602681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 184/369] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:35.603646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.628172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:35.629428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:35.741268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 186/370] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:35.742161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.772290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:35.803572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce8afd-2a98-4ff0-8810-6fd43af4c842 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:35.842934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:35.859973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a7a82-4624-4da4-8ea0-cf9838c1920b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:35.892077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:35.893051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.893333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.893631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.899058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.900068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:35.900638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.900935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:35.901221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:35.905434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:35.905581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 185/371] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:35.905658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:35.905728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:35.906783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:35.913772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 187/372] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:35.914684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:36.010439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:36.029097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b4603-5297-4c9c-bcd5-6124f75722bd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:36.082698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:36.083417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.083707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.084027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.084931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:36.085465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:36.086974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:36.093437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 188/373] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:36.098379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:36.226833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326329c-a044-4ae9-ae09-c88993bea5a2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:36.312966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647d4e8-6238-4328-8e62-c845053d5020 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:36.330790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a747b-d6c2-4920-b0f8-9b4242ae0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:36.331769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a747b-d6c2-4920-b0f8-9b4242ae0da3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.332056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a747b-d6c2-4920-b0f8-9b4242ae0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.332332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a747b-d6c2-4920-b0f8-9b4242ae0da3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.359312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:36.359907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.360192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.360470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.361405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:36.361740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:36.363841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:36.368833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a747b-d6c2-4920-b0f8-9b4242ae0da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:36.371282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 189/374] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:36 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f10a747b-d6c2-4920-b0f8-9b4242ae0da3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:36.376913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:36.377912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:36.378732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:36.380769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 186/375] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:36.384020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:36.406375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:36.481698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f227f-3e0b-45cf-88cf-466821d37f60 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.482489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f227f-3e0b-45cf-88cf-466821d37f60 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.482489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f227f-3e0b-45cf-88cf-466821d37f60 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.483883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 187/376] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:36.489792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.490433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.490933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.514824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:36.516396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:36.532592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeed193-b6ec-4ee3-98ae-ea47bb8a0665 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:36.569796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:36.614055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 188/377] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:36.622502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:36.623223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.623507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.623786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.624551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:36.624858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:36.626279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:36.639982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:36.645216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 190/378] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:36.650436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:36.745437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:36.803041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2ad5-677d-4a0e-807b-35ce61ddbc84 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:36.821185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8140eb-bf1b-4fc2-a7bd-f9cedb8da033 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:36.839002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:36.839760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.840042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.840312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.893279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:36.894032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:36.894310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:36.894580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:36.914586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:36.917494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:36.918978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:36.922377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:36.923303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:36.924138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 189/379] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:36.934760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.004390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.014006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.034022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 191/380] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:17:37.039820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.156683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b857c-f82b-4875-a3cf-a0556db089d9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.164307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecfad2-27f5-437e-a74b-57d3ba8ba07a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.211248 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06d0d-e8a5-4018-afe8-c63620675a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.212281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06d0d-e8a5-4018-afe8-c63620675a84 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.212561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06d0d-e8a5-4018-afe8-c63620675a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.212989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06d0d-e8a5-4018-afe8-c63620675a84 None admin] PATH_INFO: `/v3/projects/863bcb4c192047ed8ae210a081ac829f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.214760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06d0d-e8a5-4018-afe8-c63620675a84 None admin] RBAC: Authorizing `identity:get_project(project_id=863bcb4c192047ed8ae210a081ac829f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.216552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06d0d-e8a5-4018-afe8-c63620675a84 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.219242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 192/381] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/projects/863bcb4c192047ed8ae210a081ac829f => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:37.220480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9079c00-3054-456e-a0f9-1e72daf4cafa None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.220770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9079c00-3054-456e-a0f9-1e72daf4cafa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.221041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9079c00-3054-456e-a0f9-1e72daf4cafa None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.221781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.222707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.223156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.223574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.224472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:37.224921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.226607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6222c-0027-4561-bb5b-7a5b75143e7b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.226792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 193/382] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:37.227902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9079c00-3054-456e-a0f9-1e72daf4cafa None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.239292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 190/383] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:37.241365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.242190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.242685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.265451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:37.267390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:37.293171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.342261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 191/384] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:37.343485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.366818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9079c00-3054-456e-a0f9-1e72daf4cafa None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.368708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.409995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.410772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.411061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.411339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] PATH_INFO: `/v3/groups/bd7a783471a64c37998c40bd1eb73242` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.417260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] RBAC: Authorizing `identity:get_group(group_id=bd7a783471a64c37998c40bd1eb73242)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.418905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.421352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 194/385] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/groups/bd7a783471a64c37998c40bd1eb73242 => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:37.421566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.464527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc83b6b-3152-404b-9a86-de0344ca5c80 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.504513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.505751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.506204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.506666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.508195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.509850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.512151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 192/386] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:37.513179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.528939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2f999-e8e3-4f82-9c2a-73b7fde5a7d6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.537883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.568500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.569625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.569915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.570195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.584806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:37.595316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:37.596253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:37.597064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 195/387] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:37.597774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.618024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.630402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3179f1-bff8-4d0a-b332-b6c874e8044c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.673339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.674375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.675625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.676346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.677916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] RBAC: Adding query filter params (group.id=bd7a783471a64c37998c40bd1eb73242, role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=863bcb4c192047ed8ae210a081ac829f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:37.678455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.680128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.686662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 193/388] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/role_assignments?group.id=bd7a783471a64c37998c40bd1eb73242&scope.project.id=863bcb4c192047ed8ae210a081ac829f&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:37.687775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.730257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1aa8b-d18f-4a96-b2be-76e09048fea8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.772529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.773432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.773724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.774007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.776516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.778549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.780901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 196/389] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:37.787865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:37.832820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:37.920309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b29f65-ac0f-48ef-bb70-8bad7c336e32 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:37.962231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:37.963632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:37.963922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:37.964201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60/groups/7d307fe66c504716b2dae61eb11935ce/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:37.966843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] RBAC: Authorizing `identity:create_grant(project_id=1304cdebe410471bbff6abc40fce9b60, group_id=7d307fe66c504716b2dae61eb11935ce, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:37.968976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:37.988839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 197/390] 10.222.0.24 () {58 vars in 1442 bytes} [Thu May 19 13:17:37 2022] PUT /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60/groups/7d307fe66c504716b2dae61eb11935ce/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:17:37.995216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cb8a4-b694-4ded-baf5-11e886deba07 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:38.044610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:38.045729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:38.046208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:38.046671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:38.047695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:38.048214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:38.049921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:38.059707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 194/391] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:38.065458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:38.199872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:38.208124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd862ae-566e-48b3-bc2a-5787c57d7f13 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:38.258407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:38.258407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:38.258407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:38.258780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:38.264987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:38.266526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:38.268501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 198/392] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:38 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:38.274427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:38.312372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:38.424482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b1084-56d9-44b4-a4c8-3faf380a742d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:38.458365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6bdd-dd47-4cb7-9b04-a403bd667615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:38.459228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6bdd-dd47-4cb7-9b04-a403bd667615 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:38.459559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6bdd-dd47-4cb7-9b04-a403bd667615 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:38.459772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6bdd-dd47-4cb7-9b04-a403bd667615 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:38.460953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee6bdd-dd47-4cb7-9b04-a403bd667615 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:38.462413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee6bdd-dd47-4cb7-9b04-a403bd667615 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:38.464400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 195/393] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:38 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:38.470956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:38.633156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b590e-fb37-41ea-a137-4438a52e9309 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:38.676810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d62f6-310f-473e-abcf-c4be0b6cafd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:38.678132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d62f6-310f-473e-abcf-c4be0b6cafd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:38.678617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d62f6-310f-473e-abcf-c4be0b6cafd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:38.679089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d62f6-310f-473e-abcf-c4be0b6cafd0 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60/groups/7d307fe66c504716b2dae61eb11935ce/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:38.684156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d62f6-310f-473e-abcf-c4be0b6cafd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1304cdebe410471bbff6abc40fce9b60, group_id=7d307fe66c504716b2dae61eb11935ce, role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:38.686576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d62f6-310f-473e-abcf-c4be0b6cafd0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:38.709449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 199/394] 10.222.0.24 () {58 vars in 1442 bytes} [Thu May 19 13:17:38 2022] PUT /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60/groups/7d307fe66c504716b2dae61eb11935ce/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:38.930209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142ae0a-554e-4198-80c3-6b8e4d3e3a7a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:38.930532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142ae0a-554e-4198-80c3-6b8e4d3e3a7a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:38.930800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142ae0a-554e-4198-80c3-6b8e4d3e3a7a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:38.932644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 196/395] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:38.938835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bf62c-eb31-439f-847f-212682ac29c2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:38.939365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bf62c-eb31-439f-847f-212682ac29c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:38.939365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bf62c-eb31-439f-847f-212682ac29c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:38.963850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f49bf62c-eb31-439f-847f-212682ac29c2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:38.965247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f49bf62c-eb31-439f-847f-212682ac29c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:38.999411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:39.068405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 200/396] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:39.103840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142ae0a-554e-4198-80c3-6b8e4d3e3a7a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:39.221335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:39.239992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142ae0a-554e-4198-80c3-6b8e4d3e3a7a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:39.294256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:39.295167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:39.295451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:39.295727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:39.312480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:39.314854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641ef61-b781-4679-b9f8-04a23fae58e0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:39.315141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641ef61-b781-4679-b9f8-04a23fae58e0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:39.315416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641ef61-b781-4679-b9f8-04a23fae58e0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:39.317012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 201/397] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:39.320984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:39.321916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:39.322744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:39.323363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:39.323363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:39.323363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:39.323552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 197/398] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:39.329880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:39.347246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:39.348436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:39.460159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 202/399] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:39.487839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:39.525345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20715c5-9474-4c2c-80d0-84cb2057b5ae None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:39.600061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:39.600856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:39.601143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:39.601574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:39.602348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:39.602661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:39.604250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:39.621570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:39.624162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 198/400] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:39.630129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:39.644246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28842b5-81d5-4045-b085-4be66bb64ae3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:39.644515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:39.690986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:39.692074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:39.692506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:39.692933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:39.711862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:39.721353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:39.722525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:39.723550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:39.733479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 203/401] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:39.739851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:39.815928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8065f9-a9e2-4fde-919b-34d67d2bbdf5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:39.859892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:39.860754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:39.861037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:39.861311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:39.862998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:39.864728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:39.869319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 199/402] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:39 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:39.874322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:39.969400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:39.976295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c1ee9-e3cf-49d2-abef-b0fb60098ed0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:40.002815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6d4e-bc9c-4c90-b76f-22cf97026d27 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:40.032251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:40.032927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.033207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.033516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.034438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:40.034749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:40.036240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:40.046012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 204/403] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:40.052489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:40.058367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:40.060209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:40.061536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.061945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.062228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.063449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:40.064964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:40.071778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 200/404] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:39 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:40.072879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:40.168411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ff858-286b-4741-8066-0d647857c866 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:40.207150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:40.208119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.208403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.208762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.210109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:40.211561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:40.213209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 205/405] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:40 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:40.218806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:40.222257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e9d5-01b3-4c53-a809-5785aa2d99cd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:40.281767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:40.282658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.282937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.283214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/groups/7d307fe66c504716b2dae61eb11935ce/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.291637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4294a4df4bff4f8897d0a7b344ecbd36, group_id=7d307fe66c504716b2dae61eb11935ce, role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:40.293108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:40.316918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:40.332036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 201/406] 10.222.0.24 () {58 vars in 1442 bytes} [Thu May 19 13:17:40 2022] PUT /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/groups/7d307fe66c504716b2dae61eb11935ce/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:17:40.356447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f7b6a-3c32-4b79-abe7-7581ec15727b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:40.393718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccd4c2-e101-4b94-adf3-d579f84c30b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:40.394626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccd4c2-e101-4b94-adf3-d579f84c30b1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.394897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccd4c2-e101-4b94-adf3-d579f84c30b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.395170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccd4c2-e101-4b94-adf3-d579f84c30b1 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.396419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccd4c2-e101-4b94-adf3-d579f84c30b1 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:40.397897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccd4c2-e101-4b94-adf3-d579f84c30b1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:40.399918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 206/407] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:40 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:40.406055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:40.408942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:40.549857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280bb4f-24dd-43c8-afa7-54fe933277c6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:40.597961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421dc3a-8b92-4f70-b6bb-4b6e2df8015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:40.598855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421dc3a-8b92-4f70-b6bb-4b6e2df8015d None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.599133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421dc3a-8b92-4f70-b6bb-4b6e2df8015d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.599400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421dc3a-8b92-4f70-b6bb-4b6e2df8015d None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/groups/7d307fe66c504716b2dae61eb11935ce/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.601611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421dc3a-8b92-4f70-b6bb-4b6e2df8015d None admin] RBAC: Authorizing `identity:create_grant(project_id=4294a4df4bff4f8897d0a7b344ecbd36, group_id=7d307fe66c504716b2dae61eb11935ce, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:40.603185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421dc3a-8b92-4f70-b6bb-4b6e2df8015d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:40.631911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 202/408] 10.222.0.24 () {58 vars in 1442 bytes} [Thu May 19 13:17:40 2022] PUT /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36/groups/7d307fe66c504716b2dae61eb11935ce/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:40.809363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497dfb6-64ee-42ee-91cb-918a48e17f80 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.809576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497dfb6-64ee-42ee-91cb-918a48e17f80 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.809847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497dfb6-64ee-42ee-91cb-918a48e17f80 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.811483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 207/409] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:40.817170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea16ca-41a3-4944-95de-def28182537a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:40.817804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea16ca-41a3-4944-95de-def28182537a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:40.818310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea16ca-41a3-4944-95de-def28182537a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:40.836411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bfea16ca-41a3-4944-95de-def28182537a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:40.837987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bfea16ca-41a3-4944-95de-def28182537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:40.917155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:40.933526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 203/410] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:40.951082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497dfb6-64ee-42ee-91cb-918a48e17f80 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.078343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497dfb6-64ee-42ee-91cb-918a48e17f80 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:41.123024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:41.123964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.124257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.124536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.136346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:41.141655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:41.149228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:41.150143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:41.150948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 208/411] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:41.156596 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea16ca-41a3-4944-95de-def28182537a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.285829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea16ca-41a3-4944-95de-def28182537a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:41.321769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:41.322459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.322742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.323016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.323799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:41.324102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:41.325595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:41.335784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 204/412] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:41.341706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.468238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a65e2-c993-44b9-b4da-3b42e0a6373b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:41.509653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d132a-7d10-4707-8338-89d0b199b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:41.510514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d132a-7d10-4707-8338-89d0b199b10d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.510791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d132a-7d10-4707-8338-89d0b199b10d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.511062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d132a-7d10-4707-8338-89d0b199b10d None admin] PATH_INFO: `/v3/groups/bd7a783471a64c37998c40bd1eb73242` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.512228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d132a-7d10-4707-8338-89d0b199b10d None admin] RBAC: Authorizing `identity:get_group(group_id=bd7a783471a64c37998c40bd1eb73242)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:41.513785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378d132a-7d10-4707-8338-89d0b199b10d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:41.515743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 209/413] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:41 2022] GET /identity/v3/groups/bd7a783471a64c37998c40bd1eb73242 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:41.521347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.522467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450429d2-13b2-4e81-8299-fd8cb0652935 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.522732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450429d2-13b2-4e81-8299-fd8cb0652935 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.523013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450429d2-13b2-4e81-8299-fd8cb0652935 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.524593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 210/414] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:41.526525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0a8da-edd4-4e98-8f22-c543c12e6af9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.526813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0a8da-edd4-4e98-8f22-c543c12e6af9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.527091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0a8da-edd4-4e98-8f22-c543c12e6af9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.528399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 211/415] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:17:41.533494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272647ee-01a4-4e5e-9213-4a16f8796ce4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.533788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272647ee-01a4-4e5e-9213-4a16f8796ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.534069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272647ee-01a4-4e5e-9213-4a16f8796ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.551994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-272647ee-01a4-4e5e-9213-4a16f8796ce4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:41.553186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-272647ee-01a4-4e5e-9213-4a16f8796ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:41.589378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:41.627119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:41.637431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 212/416] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:41.640109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.645768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.646208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.652642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b343e5-5a73-4d10-b9ed-1451db8c9024 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:41.676106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:41.677751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:41.693811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:41.694688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.694998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.695709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] PATH_INFO: `/v3/projects/863bcb4c192047ed8ae210a081ac829f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.697184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] RBAC: Authorizing `identity:get_project(project_id=863bcb4c192047ed8ae210a081ac829f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:41.698921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:41.701818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 205/417] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:41 2022] GET /identity/v3/projects/863bcb4c192047ed8ae210a081ac829f => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:41.702637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.758889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 213/418] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:41.760106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.820879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22358f94-150b-4b5d-a5fd-0db8bee9f6b3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:41.844538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:41.857659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:41.858657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.859105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.859520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.869961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:41.874814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795344-64a5-4489-aecd-814c6f915c54 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:41.878485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:41.879537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:41.880368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:41.882825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 206/419] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:41.883789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:41.911635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:41.923065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:41.924392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:41.925182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:41.925943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] PATH_INFO: `/v3/projects/863bcb4c192047ed8ae210a081ac829f/groups/bd7a783471a64c37998c40bd1eb73242/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:41.928503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=863bcb4c192047ed8ae210a081ac829f, group_id=bd7a783471a64c37998c40bd1eb73242, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:41.930650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:41.953435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 214/420] 10.222.0.24 () {58 vars in 1442 bytes} [Thu May 19 13:17:41 2022] PUT /identity/v3/projects/863bcb4c192047ed8ae210a081ac829f/groups/bd7a783471a64c37998c40bd1eb73242/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:17:41.953435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:42.069691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3573-5967-4be9-9b58-def5fb38861b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:42.114575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:42.115784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:42.116291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:42.116761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:42.130962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:42.140150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:42.141124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:42.141968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:42.144767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 207/421] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:42.151070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:42.192443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e98416-aaed-473a-b86d-dc509cefb7e4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:42.232180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:42.251742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:42.252697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:42.253130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:42.253589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:42.254497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:42.254954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:42.262051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:42.271768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 215/422] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:42.277657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:42.295349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088745e1-f6ac-4a5b-81e5-8cefa00b0da6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:42.333021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:42.333876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:42.334262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:42.334718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:42.335612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] RBAC: Adding query filter params (name=member) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:42.336072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:42.337652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:42.339732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:42.348772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 208/423] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:42.354707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:42.448239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918a6d7-a5aa-41a7-a00f-68d2523d0730 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:42.503901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3145f5e-dda7-4591-8bc2-322071b4b837 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:42.507620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:42.528774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:42.529684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:42.529902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:42.530174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:42.552433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:42.553455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:42.553957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:42.554422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:42.611854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:42.613787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:42.669327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:42.732626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 216/424] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:42.738447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:42.741379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:42.743048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:42.909271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed20d6-8c48-443e-be29-eded35777ed7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:42.920052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 209/425] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:42.921281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:42.926202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:42.976219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:42.977366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:42.977814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:42.978231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:42.980223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:42.981972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:42.989102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 217/426] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:42 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:42.996372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.027167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.111664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f466e-2023-44b6-9c8e-f67dc825690d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.178677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.179563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.179839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.180109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] PATH_INFO: `/v3/projects/1304cdebe410471bbff6abc40fce9b60` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.181455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] RBAC: Authorizing `identity:get_project(project_id=1304cdebe410471bbff6abc40fce9b60)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:43.183044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:43.185155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 210/427] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:42 2022] GET /identity/v3/projects/1304cdebe410471bbff6abc40fce9b60 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:43.188462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f49b44-d213-4a3b-bb1b-3f7a47e7ed6a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.191048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.235928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.237552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.238158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.238635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.240615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:43.242464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:43.244755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 218/428] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:42 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:43.252107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.306492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.339979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd4f22-3057-4913-aba5-3b6566d6b964 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.358162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.379944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404779cb-495d-4154-a207-ed71a58af705 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.389946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eef093-4bc0-4010-8d6a-b8110c997798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.391385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef093-4bc0-4010-8d6a-b8110c997798 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.391827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef093-4bc0-4010-8d6a-b8110c997798 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.392546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eef093-4bc0-4010-8d6a-b8110c997798 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.394194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eef093-4bc0-4010-8d6a-b8110c997798 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:43.397467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eef093-4bc0-4010-8d6a-b8110c997798 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:43.400639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 211/429] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:43.402962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d55e-e805-4de8-be1b-32d13a20585e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.403407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d55e-e805-4de8-be1b-32d13a20585e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.404201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d55e-e805-4de8-be1b-32d13a20585e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.410613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 212/430] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:43.411286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e170c-09b3-46dd-8c4e-2ab2ea364412 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.411806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e170c-09b3-46dd-8c4e-2ab2ea364412 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.412082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e170c-09b3-46dd-8c4e-2ab2ea364412 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.414553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 213/431] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:43.415409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e170c-09b3-46dd-8c4e-2ab2ea364412 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.420619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.421285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.421595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.421868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.422705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=1304cdebe410471bbff6abc40fce9b60) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:43.423020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:43.424461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6be07-26ad-4250-b5e7-7a62b5f87c74 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:43.431058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 219/432] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:43.432258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ae0c2-18b8-4ebb-a6ca-b2b83f49d646 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.432543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ae0c2-18b8-4ebb-a6ca-b2b83f49d646 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.432869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ae0c2-18b8-4ebb-a6ca-b2b83f49d646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.450442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a95ae0c2-18b8-4ebb-a6ca-b2b83f49d646 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:43.452032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a95ae0c2-18b8-4ebb-a6ca-b2b83f49d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:43.526214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 220/433] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:43.527767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.528191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.528632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.560337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:43.562062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:43.573726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e170c-09b3-46dd-8c4e-2ab2ea364412 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.621753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.623224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.623515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.623791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.624912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=1304cdebe410471bbff6abc40fce9b60) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:43.625220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:43.627205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:43.641513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 214/434] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=1304cdebe410471bbff6abc40fce9b60&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:43.642516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.647951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 221/435] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:43.653932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.676796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.692714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.784444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f217383-79b1-4980-b11b-da776a14ea40 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.795083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327ee37-01e2-44aa-8775-489a93d6af33 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.828496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.829744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.830232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.830714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.832871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:43.833713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:43.833998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:43.834275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:43.844105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:43.846996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:43.852190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:43.853188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:43.854088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 215/436] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:43.855277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:43.856125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:43.857057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 222/437] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:43.861706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.862566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:43.934729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.943362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:43.979895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cca1e2-bb95-41fc-bdb6-169d93eec5af None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:43.987836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b5c25-782b-4d1e-99a0-936ec89a9e21 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.026865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.027859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.028154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.028524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.029022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.029116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.029197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.029274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.029917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:44.030034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:44.030224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.030324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.031611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.031709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.041622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 216/438] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:44.042615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 223/439] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:17:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:44.048162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.049925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.172509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde091-1cd5-4cb6-bbad-6c40900c2474 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.172920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885bc07-ff90-4e99-a2b9-52800a4928a5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.208105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.214757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.224212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.224412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.224876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.224994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.225160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.225256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.225445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.225549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.311099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.312584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.325159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.326925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.445351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 224/440] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:44.448052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 217/441] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:44.453055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.453210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.479957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.486679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.557218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829f1ae-b49c-4017-bae5-6c0fe4c30977 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.557503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1511976-8469-4c6b-af48-313917c25d35 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.590055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722abf6-fe40-4758-adff-33eec4d0c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.590370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.590838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722abf6-fe40-4758-adff-33eec4d0c180 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.591186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722abf6-fe40-4758-adff-33eec4d0c180 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.591392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.591608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722abf6-fe40-4758-adff-33eec4d0c180 None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.591896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.592263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] PATH_INFO: `/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.592793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722abf6-fe40-4758-adff-33eec4d0c180 None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.593477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] RBAC: Authorizing `identity:get_project(project_id=4294a4df4bff4f8897d0a7b344ecbd36)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.594263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722abf6-fe40-4758-adff-33eec4d0c180 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.594949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.595956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 218/442] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:44.596815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 225/443] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/projects/4294a4df4bff4f8897d0a7b344ecbd36 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:44.597249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0405-5667-416b-b8d6-065acf9e3835 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.597563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0405-5667-416b-b8d6-065acf9e3835 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.597896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a0405-5667-416b-b8d6-065acf9e3835 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.599214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 219/444] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:44.601584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.601755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0405-5667-416b-b8d6-065acf9e3835 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.696870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.701053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.704537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87d429-d88a-43ca-973f-2686f28a6a2b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.706692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a0405-5667-416b-b8d6-065acf9e3835 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.738921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e41ee-c802-4856-b0d0-d8bf6ba3c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.739772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e41ee-c802-4856-b0d0-d8bf6ba3c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.739888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.740055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e41ee-c802-4856-b0d0-d8bf6ba3c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.740329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e41ee-c802-4856-b0d0-d8bf6ba3c4e2 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.740702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.740980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.741246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] PATH_INFO: `/v3/groups/7d307fe66c504716b2dae61eb11935ce` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.741533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e41ee-c802-4856-b0d0-d8bf6ba3c4e2 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.742421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] RBAC: Authorizing `identity:get_group(group_id=7d307fe66c504716b2dae61eb11935ce)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.742996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e41ee-c802-4856-b0d0-d8bf6ba3c4e2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.743829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.744619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 226/445] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:44.745856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.746138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.746410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.748920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 220/446] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/groups/7d307fe66c504716b2dae61eb11935ce => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:44.750565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.769320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:44.770530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:44.836991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 227/447] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:44.837841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.857189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3ec2b-e24e-4ed4-8007-e41e0684c1f1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.891842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.892494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.892774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.893067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.893938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:44.894231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.895660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.901442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 221/448] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:44.902207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:44.940685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.940903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:44.947921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af9a5e-fced-430b-8e8d-ca687d84c097 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:44.986872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:44.987890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:44.988155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:44.988422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:44.989274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] RBAC: Adding query filter params (group.id=7d307fe66c504716b2dae61eb11935ce, role.id=cc93bbcc688e4a08879b0b936b6018ea, scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:44.989617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:44.991185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:44.997916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 228/449] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/role_assignments?group.id=7d307fe66c504716b2dae61eb11935ce&scope.project.id=4294a4df4bff4f8897d0a7b344ecbd36&role.id=cc93bbcc688e4a08879b0b936b6018ea => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:45.034273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52bd6-826a-48b6-9fd1-d9bd0e631894 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:45.073504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:45.074324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:45.074600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:45.074866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:45.087347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:45.095307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:45.096372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:45.097248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 222/450] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:45.104341 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:45.217946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6f6ef-b099-412a-be7a-42eb11846956 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:45.251760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:45.252435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:45.252711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:45.252981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:45.253786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:45.254094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:45.255508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:45.265333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 229/451] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:45.271073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:45.358428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:45.376821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62252-40e7-48f6-b0b5-1aa70f32ef06 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:45.410453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:45.411097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:45.411375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:45.411645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:45.480290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:45.481659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:45.568049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:45.584303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 223/452] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:45.590894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:45.624108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:45.711096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0112b6-1610-4a9f-a9e9-48c193d0c6c2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:45.749145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:45.750319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:45.750593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:45.750871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] PATH_INFO: `/v3/projects/863bcb4c192047ed8ae210a081ac829f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:45.752168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] RBAC: Authorizing `identity:get_project(project_id=863bcb4c192047ed8ae210a081ac829f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:45.753626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:45.755303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 230/453] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:17:45 2022] GET /identity/v3/projects/863bcb4c192047ed8ae210a081ac829f => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:45.761225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:45.881174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c604bc-1a4c-4b8c-b63e-8dc0fdc6c755 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:45.924824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba31da9-c2c2-4301-b9f0-0e924471d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:45.925698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba31da9-c2c2-4301-b9f0-0e924471d651 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:45.925973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba31da9-c2c2-4301-b9f0-0e924471d651 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:45.926236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba31da9-c2c2-4301-b9f0-0e924471d651 None admin] PATH_INFO: `/v3/groups/bd7a783471a64c37998c40bd1eb73242` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:45.927519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba31da9-c2c2-4301-b9f0-0e924471d651 None admin] RBAC: Authorizing `identity:get_group(group_id=bd7a783471a64c37998c40bd1eb73242)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:45.928947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba31da9-c2c2-4301-b9f0-0e924471d651 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:45.930568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 224/454] 10.222.0.24 () {56 vars in 1180 bytes} [Thu May 19 13:17:45 2022] GET /identity/v3/groups/bd7a783471a64c37998c40bd1eb73242 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:45.937245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:46.037923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:46.049034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fb346-9232-463c-97ca-b8a152e68635 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:46.085053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:46.085734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:46.086019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:46.086290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:46.087117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] RBAC: Adding query filter params (group.id=bd7a783471a64c37998c40bd1eb73242, role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=863bcb4c192047ed8ae210a081ac829f) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:46.087432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:46.088864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7d018-b285-4855-b12a-b05953fb669c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:46.095199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 231/455] 10.222.0.24 () {56 vars in 1376 bytes} [Thu May 19 13:17:45 2022] GET /identity/v3/role_assignments?group.id=bd7a783471a64c37998c40bd1eb73242&scope.project.id=863bcb4c192047ed8ae210a081ac829f&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:48.717351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85800498-7580-46ea-bd09-5667813d8f58 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:48.717885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85800498-7580-46ea-bd09-5667813d8f58 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:48.717885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85800498-7580-46ea-bd09-5667813d8f58 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:48.719402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 225/456] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:48.734850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742912-df52-44b8-9eb5-9b3dfe448859 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:48.735699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742912-df52-44b8-9eb5-9b3dfe448859 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:48.736184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2742912-df52-44b8-9eb5-9b3dfe448859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:48.762348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d2742912-df52-44b8-9eb5-9b3dfe448859 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:48.764204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d2742912-df52-44b8-9eb5-9b3dfe448859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:48.855506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 232/457] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:48.901352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85800498-7580-46ea-bd09-5667813d8f58 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:48.984219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.031542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6bc17-b66e-45ae-b105-67f961cbbe6f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.032001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6bc17-b66e-45ae-b105-67f961cbbe6f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.032566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6bc17-b66e-45ae-b105-67f961cbbe6f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.035015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 233/458] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:17:49.046244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85800498-7580-46ea-bd09-5667813d8f58 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.056443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d060c1b-e79e-4dec-be95-b1fb746bc5e7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.056871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d060c1b-e79e-4dec-be95-b1fb746bc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.057280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d060c1b-e79e-4dec-be95-b1fb746bc5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.091135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6d060c1b-e79e-4dec-be95-b1fb746bc5e7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:49.092353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6d060c1b-e79e-4dec-be95-b1fb746bc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:49.099289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.100097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.100376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.100651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.115173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.116925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-08963b39-e8f5-473c-b624-ba6f2e4d052d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:49.139334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:49.140490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:49.141594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:49.141594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:49.141594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:49.141594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:49.143549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 226/459] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:49.145966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd4f9c-e19d-4d49-9ac9-098efb01226e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.146468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd4f9c-e19d-4d49-9ac9-098efb01226e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.146968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd4f9c-e19d-4d49-9ac9-098efb01226e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.148657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 227/460] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:49.149562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd4f9c-e19d-4d49-9ac9-098efb01226e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.180553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 234/461] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:49.182430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.182887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.183160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.212288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:49.213883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:49.229403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.290489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd4f9c-e19d-4d49-9ac9-098efb01226e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.298828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.321233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 235/462] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:49.322326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.346657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.347322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.347669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.347876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.348604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:49.348911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.354062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.366826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 228/463] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:49.367795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.447115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed56c8-a04b-4c09-958d-18d0cd8c7373 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.487301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc0513-8b45-44be-9b19-7941f708d297 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.493909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.495389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.495822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.496290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.511573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.513975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.521622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.526001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.527128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.527618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.528175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:49.535348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:49.536220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:49.537110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:49.537110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:49.537110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:49.537110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:49.541748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 236/464] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:49.542608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.544183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.547389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:49.570790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:49.571730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:49.572615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:49.572615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:49.572615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:49.572615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:49.575048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 229/465] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:49.576223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.601524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.660861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8f5d-f68d-44e1-83bd-c0120181b0d1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.689440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a179c-aace-4b4c-a483-640e931294e9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.697038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.697953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.698393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.698671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.699386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.701104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.724552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 237/466] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:49 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:49.725374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.727836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.728592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.728938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.729274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.730107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:49.730477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.731987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c874d-6c9c-40f6-a9d6-14139ae774b6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.748735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 230/467] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:49.750514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa650f-09ae-4887-9838-5849c9d90131 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.750801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa650f-09ae-4887-9838-5849c9d90131 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.751081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa650f-09ae-4887-9838-5849c9d90131 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.753527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 231/468] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:49.754519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa650f-09ae-4887-9838-5849c9d90131 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:49.807775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.864947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f71be0-bb79-403f-86fb-76d386452ab3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.876711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:49.907728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.908744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa650f-09ae-4887-9838-5849c9d90131 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:49.909055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.909567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.910038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.911095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:49.911731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.913402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646b874-147e-42e6-90fc-131693a1dc54 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.935533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 238/469] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:49.937312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.937877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.938392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.955643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:49.956330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:49.956607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:49.956879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:49.957622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:49.959772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:49.961248 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:49.963204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:49.980491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 232/470] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:49 2022] POST /identity/v3/users => generated 248 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:49.986371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.057536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 239/471] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:50.088267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.126270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197dc46-a2fc-4d6e-9465-cb17f18b91cc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:50.148828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:50.176941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3bc46-37b3-4397-8d08-57896e197078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:50.177910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bc46-37b3-4397-8d08-57896e197078 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.178200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bc46-37b3-4397-8d08-57896e197078 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.178473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3bc46-37b3-4397-8d08-57896e197078 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.179190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bc46-37b3-4397-8d08-57896e197078 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:50.181557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3bc46-37b3-4397-8d08-57896e197078 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:50.190549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:50.217374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 233/472] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:49 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:50.228855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575400c5-ebc6-4c66-a2e5-ff3afede4cd4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:50.267687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:50.268505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.268782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.269057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.275843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea4597-1a2c-4d34-9cc8-084c5d95e647 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.276153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea4597-1a2c-4d34-9cc8-084c5d95e647 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.276406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea4597-1a2c-4d34-9cc8-084c5d95e647 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.279525 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 234/473] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:17:50.285825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.286245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.286428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:50.286704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.288271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:50.311159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:50.312265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:50.313374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:50.313374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:50.313374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:50.313374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:50.314036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:50.315490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:50.315656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 240/474] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:50.321134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.402528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 235/475] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:50.434019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.440749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:50.449152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297d34d-5497-4463-81a0-60870451fb15 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:50.486775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:50.487690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.488110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.488550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.489435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:50.489879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:50.491406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:50.504448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 241/476] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:50.510206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.574682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5965d2-8d50-4c5c-9b27-bfb1bd895e68 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:50.602677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:50.622387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:50.624077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.624322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.624610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.625669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:50.627409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:50.630607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e51a-6147-426c-92ff-a02b83126294 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:50.635482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 13:17:50.635482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 13:17:50.635482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [parameters: {'id': 'd127445d27704e20a9f898aede78e3fa', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 13:17:50.635482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() May 19 13:17:50.657334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return ref.to_dict() May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:17:50.658581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 13:17:50.659722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 13:17:50.660830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:17:50.662648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:17:50.664854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': 'd127445d27704e20a9f898aede78e3fa', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:50.666822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 13:17:50.668734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:50.670968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 236/477] 10.222.0.24 () {60 vars in 1131 bytes} [Thu May 19 13:17:50 2022] POST /identity/v3/roles => generated 144 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:50.670968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.679713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:50.680398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.680678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.680960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.681698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:50.683150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:50.715076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 242/478] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:50 2022] POST /identity/v3/users => generated 249 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:50.837727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f210-e576-4155-8c6d-f4fa87d6bdca None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:50.873966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a6d7-dc0d-4cab-ba35-99c856813dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:50.874799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a6d7-dc0d-4cab-ba35-99c856813dfa None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:50.875081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a6d7-dc0d-4cab-ba35-99c856813dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:50.875356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a6d7-dc0d-4cab-ba35-99c856813dfa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:50.887242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a6d7-dc0d-4cab-ba35-99c856813dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b686a6d7-dc0d-4cab-ba35-99c856813dfa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:50.895574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:50.896587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:50.897422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 237/479] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:17:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:50.902849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:50.940548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:51.041195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4646f-bc61-4f49-b2ee-fffee6037410 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:51.079085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:51.080016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:51.080446 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:51.080886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:51.081887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:51.082360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:51.083904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f54cf-4041-44fc-b423-c304268a6a14 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:51.093910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 243/480] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:52.667096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2d89c-0f39-48ef-b09b-afce2bb769e8 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:52.668132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2d89c-0f39-48ef-b09b-afce2bb769e8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:52.668621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2d89c-0f39-48ef-b09b-afce2bb769e8 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:52.670995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 238/481] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:52.676389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763efb2-7a4c-4fbd-95dd-683dfde4c699 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:52.677279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763efb2-7a4c-4fbd-95dd-683dfde4c699 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:52.677791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763efb2-7a4c-4fbd-95dd-683dfde4c699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:52.702028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3763efb2-7a4c-4fbd-95dd-683dfde4c699 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:52.703574 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3763efb2-7a4c-4fbd-95dd-683dfde4c699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:52.796123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 244/482] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:52.831152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2d89c-0f39-48ef-b09b-afce2bb769e8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:52.891401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:52.923405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32297912-9c6b-4651-889a-6658c02088db None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:52.924014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32297912-9c6b-4651-889a-6658c02088db None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:52.924506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32297912-9c6b-4651-889a-6658c02088db None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:52.926372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 245/483] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:52.931248 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:52.931774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:52.932249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:52.941260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:52.954880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:52.956092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:52.989954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2d89c-0f39-48ef-b09b-afce2bb769e8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.054655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.055813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.056310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.056767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.082386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:53.086521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 246/484] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:53.093436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:53.094379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:53.095200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:53.097079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 239/485] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:53.103957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.119589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.211546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df0591-1529-4567-b3ca-1aab4ca9ea3b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.257393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.258097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.258384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.258664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.259432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:53.259744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:53.261208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719839-7e22-4bbf-8bc6-e28d4ce2ba46 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:53.269450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:53.275162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce3398-ad68-4c30-847c-be015d702f31 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.279865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 247/486] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:53.279865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92584-c79e-49cb-ad03-84d099b2d5ba None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.279865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92584-c79e-49cb-ad03-84d099b2d5ba None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.279865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c92584-c79e-49cb-ad03-84d099b2d5ba None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.284109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 248/487] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:53.285603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92584-c79e-49cb-ad03-84d099b2d5ba None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.286014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:53.322426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37eeeed-72fe-4895-a4ab-fdb9e15ad8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.323217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37eeeed-72fe-4895-a4ab-fdb9e15ad8dd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.323663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37eeeed-72fe-4895-a4ab-fdb9e15ad8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.324077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37eeeed-72fe-4895-a4ab-fdb9e15ad8dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.335281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37eeeed-72fe-4895-a4ab-fdb9e15ad8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d37eeeed-72fe-4895-a4ab-fdb9e15ad8dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:53.343783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:53.344820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:53.345695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:53.352886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 240/488] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:53.354378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.354841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.355344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.373497 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:53.374978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:53.427658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c92584-c79e-49cb-ad03-84d099b2d5ba None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.456697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 241/489] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:53.465518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.466926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.467509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.468046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.468269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.480248 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:53.482135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:53.500361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:53.501285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:53.502218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:53.502218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:53.502218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:53.502218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:53.509564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 249/490] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:17:53.511377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.564528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:53.594472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:53.602418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b91a-650d-476f-9949-5855daa78489 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.620004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80010397-3418-4f34-a336-b8f408c882d5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.639001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.639675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.639963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.640240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.641021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:53.641514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:53.642706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:53.651413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 242/491] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:53.652450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.659467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.660335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.660758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.661175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.662090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:53.662568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:53.664301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc43570-9a99-428e-b4a8-620d373e0390 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:53.681074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 250/492] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:17:53.681074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c286af-15d7-4d96-ba17-e4f34f7c4ab7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.681074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c286af-15d7-4d96-ba17-e4f34f7c4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.681074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c286af-15d7-4d96-ba17-e4f34f7c4ab7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.681074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 251/493] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:53.681074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c286af-15d7-4d96-ba17-e4f34f7c4ab7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.770455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186d00f-316f-4b4a-ad93-9c0596075e35 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.794873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c286af-15d7-4d96-ba17-e4f34f7c4ab7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.804483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:53.806898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.807700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.807983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.808259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.819335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:53.827089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:53.828480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 243/494] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:53.829818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:53.840520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:53.841551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.841921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.842222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.853128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:53.854691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:53.859391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0a4eb66b-8a56-4268-af67-152ed2ac6f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:53.873290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:53.874564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:53.875457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:53.875457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:53.875457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:53.875457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:53.875457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 252/495] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:53.876323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:53.876608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:53.876879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:53.893860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:53.895057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:53.955575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6566a-3ada-4d7b-a291-b39f5c31cb0d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:53.962860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 253/496] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:53.963690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.001940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.002738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.003018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.003292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.011989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.013634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7f96a23d-0285-4ff0-b958-f19c083885eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:17:54.029284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:54.030272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:54.031204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 244/497] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:54.031788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cdd14-373c-41d1-b5a4-4a1de102f86a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.031908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cdd14-373c-41d1-b5a4-4a1de102f86a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.032185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cdd14-373c-41d1-b5a4-4a1de102f86a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.033464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 245/498] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:54.034155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cdd14-373c-41d1-b5a4-4a1de102f86a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.059973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.070188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21480b77-32a8-4a45-be99-51384ce58c34 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.102529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.103190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.103471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.103745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.104559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:54.104817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.106207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.111112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.114979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 254/499] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:17:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:54.115743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.135414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cdd14-373c-41d1-b5a4-4a1de102f86a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.169305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.170080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.170402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.171034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.171389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:54.171692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.173066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.184738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 246/500] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:54.186072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.220267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707bb2f-362d-4768-a7e6-3fa7ab0709db None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.256245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a22dea-92fd-4b3c-a13d-d6201e9d8871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.257055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a22dea-92fd-4b3c-a13d-d6201e9d8871 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.257338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a22dea-92fd-4b3c-a13d-d6201e9d8871 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.257647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a22dea-92fd-4b3c-a13d-d6201e9d8871 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.269977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a22dea-92fd-4b3c-a13d-d6201e9d8871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-89a22dea-92fd-4b3c-a13d-d6201e9d8871 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:54.277745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:54.278744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:54.279633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 255/501] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:54.280943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.280943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.280943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.297284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:54.298501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:54.298928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807510-8d89-4650-b040-afd4ba60c2b1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.306242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.333750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.334419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.334701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.334972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.335812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:54.336117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.337569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.348520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.367261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 256/502] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:54.368097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.377116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 247/503] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:54.377970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.473780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2c302-9413-4759-b465-fface4352ce1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.473996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429a880-61cf-4c2e-8e04-f2333b3bd9cf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.507959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.508754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.509035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.509314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.511876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.512660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.513169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.513706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.516810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.518279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.524354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.525876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.527655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:17:54.533498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:54.534498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 248/504] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:54.535749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:54.550769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:54.551754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:54.552730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:54.552730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:54.552730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:54.552730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:54.552730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 257/505] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:54.553120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.596542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.642878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a62ee-26c4-4bc3-b6e2-014747b7a1f3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.654830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83191b8-c8cd-4bb3-bfba-a379390f57de None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.676733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.677392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.677695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.677966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.678722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:54.679025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.680394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.688328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 249/506] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:54.689107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.689748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.690400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.690686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.690960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.750893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.758760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.760227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.785435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f465f1c-1b83-4e0b-9c8d-a6398955d35b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.801367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.820251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.821045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.821326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.821625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.832897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.834378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:54.851291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:54.852168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:54.853059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:54.853059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:54.853059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:54.853059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:54.853059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 250/507] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:54.853335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.864214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 258/508] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:54.865051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:54.957043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:54.961178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263c426-930e-4306-aa9d-b9a196c8da97 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.964111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547de9f-e0f3-454f-b4b1-4bd6bce444cc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:54.994810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.995473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.995756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.996028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.996871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:54.997203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:54.997896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:54.998550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:54.998678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:54.998829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:54.999101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:54.999812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:55.000118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.001490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.012680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 259/509] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:55.013724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.020188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 251/510] 10.222.0.24 () {56 vars in 1137 bytes} [Thu May 19 13:17:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:55.020188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.046346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.115921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05ed9f-e876-408d-87f1-c866fcce5c58 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.118161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ac1a5-9bb2-4ce7-90d4-0c42a4f36fcc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.148604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.149395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.149708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.149984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.150977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.151627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.151909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.152180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.152901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:55.153205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.155087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.160621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.162126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.166593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 252/511] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:55.167412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:55.179837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:55.180947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:55.181888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:55.181888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:55.181888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:55.181888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:55.181888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 260/512] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:55.181888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.192400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.267505 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.282824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7f177-d2e8-4bf4-83c6-63f8fd802269 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.285588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea9053-4bc3-4ce7-89d6-9e52e84925ac None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.317966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.318442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.318692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.318977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.319243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.319347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.319526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.319803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.326754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.328219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.330938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.332401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:17:55.343582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:55.344504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 261/513] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:55.345696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:55.349774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:55.350820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:55.351810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:55.351810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:55.351810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:55.351810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:55.355266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 253/514] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:55.356070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.412515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.448332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48691927-493d-4584-be3c-964ac3fda22a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.460426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8a2f-4850-407c-9dbf-9cf9e06d4f98 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.482095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.482838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.483120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.483403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.492529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.493186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.493491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.493789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.494560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.496013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.504912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.517989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 254/515] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:17:55 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:17:55.518809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.565022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.566579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.639993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.644406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b90a0e-d0aa-4781-ae60-35390fbcd594 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.678782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 262/516] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:55.679992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.688826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.689912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.690438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.690779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.691536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:55.691900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.693314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.706743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 255/517] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:55.707661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.770593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.798887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787a7f1-be53-4403-bc8a-3f8a81b85167 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.820371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776320cd-8585-4fbd-bd4e-3283a194bbd5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.834668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.835585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.835585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.835902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.836740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:55.837049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.838514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.855791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:55.856544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:55.857021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:55.857601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:55.858187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:55.858495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:55.859867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:55.876491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 256/518] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:55.876491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.885259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 263/519] 10.222.0.24 () {56 vars in 1137 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:55.885259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:55.910559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:55.986130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865eab1c-8eab-4ddb-be70-63d6384fc8f4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:55.994425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfb1d2-9391-45bc-9d23-69d6d5365793 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.002251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.025396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.026225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.026520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.026795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.031733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.032735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.033185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.033660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.041833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.042007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.043299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.043398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:17:56.059017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:56.062226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:56.063161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 264/520] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:56.063744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:56.064553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:56.065482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:56.065482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:56.065482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:56.065482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:56.065732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.065810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 257/521] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:56.065810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.155066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.170601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea225-cd54-4add-97de-3f89de8aadee None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.173361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986fd8-ec14-480c-b80d-b107cdc4f5dc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.205612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.206290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.206568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.206845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.208855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.209836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.210885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.211178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.218366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.219826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:17:56.235113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:56.235992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:56.236874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 265/522] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:56.238077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.251766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.294059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.295601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.354710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1f3a0-56fe-4035-9525-0329681a2a0f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.391910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.392951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.393243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.393684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.394535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:56.394849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.396310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.402738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 258/523] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:56.404124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.419546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 266/524] 10.222.0.24 () {56 vars in 1139 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:56.420442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.423698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.472890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.509199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f34b-5047-4baf-a59a-2d807f8d3a45 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.542820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8c058-ffae-4805-a067-724af7a05c6b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.553602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.554273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.554551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.554862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.555547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:56.555848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.557228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.573799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 259/525] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:56.573799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.585974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.586786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.587072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.587351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.588146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:56.588458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.590046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.612482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 267/526] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:56.612482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.674944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.685454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1abb6-637e-4bda-aa09-093a63a56c88 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.712773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.718478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fadc402-5aa5-4b44-875c-e69a93baf767 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.723870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.724659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.724937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.725206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.736778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.738245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.753238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.753918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.754203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.754480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:56.755595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:56.756464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:56.757928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:56.757928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:56.757928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:56.757928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:56.757928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 260/527] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:56.757928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.829926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.831397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.872194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e421b48-4a11-49ed-bf1c-875fc07255e8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:56.896056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:56.907403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:56.908203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:56.908481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:56.908753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:56.915852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:56.917306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:17:56.932531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:56.933616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 261/528] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:56.938132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.949720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 268/529] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:17:56.951194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:56.980384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.047815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4a0b6-7870-416a-b173-d1a80ef17dcf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.051839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c635e-9c45-42ab-9b77-f16466d905fb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.084361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.085015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.085293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.085629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.085887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.086458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=862f9782f8574b8bb99d38c51333277f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.086566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.086766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.086857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.087071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.087793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.088096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.088194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.089473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.101785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 262/530] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/role_assignments?user.id=862f9782f8574b8bb99d38c51333277f&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.101785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.113928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 269/531] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.115052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.141796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.230485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509dbf-761a-4721-87e4-9d09fee5ddcd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.233902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f25f8f-55f8-4e62-b6cd-23c0f3a1a562 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.245815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.281063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.281918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.282211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.282493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.283431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.284756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.285279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.287505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.288917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.289397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.289947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.304176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.307031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.313825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 263/532] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.314773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:57.330025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:57.330950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:57.331829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:57.331829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:57.331829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:57.331829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:57.332465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 270/533] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:57.333102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.422513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e78d4-2c58-4363-8a6c-03bed9e7b324 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.433771 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.462704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.463952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.464427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.464913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.466356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a6c3b-002f-4262-b5c7-ccf4d9025ae7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.472756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.474609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:17:57.490839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:57.491962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:57.493062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:57.494912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 264/534] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:57.495909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.516708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.517651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.518070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.518503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.520003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=d92108b779ca47aca8f941b7dfae80d5) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.520461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.522601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.523981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.533044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 271/535] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/role_assignments?user.id=d92108b779ca47aca8f941b7dfae80d5&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.534099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.662292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4f84c-4087-465b-b9e0-98e710454fc5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.700909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.701872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.702569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.703015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.704138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.704594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.706819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.717942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.721985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 272/536] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.724147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906ed7e-fbc0-4ff3-b92b-1b282848f196 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.728600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.795917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.796500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.796785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.797063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.797844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.798150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.799645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8537c-c38f-4d87-b7de-14278f524c88 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.814768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 265/537] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.816139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6abaed-d35a-425d-979b-812b58861d2b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.816436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6abaed-d35a-425d-979b-812b58861d2b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.816707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6abaed-d35a-425d-979b-812b58861d2b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.822257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 266/538] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:17:57.823216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6abaed-d35a-425d-979b-812b58861d2b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:57.872187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6ee25-79fe-429e-9a2a-658234e30092 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.913361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:57.914912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.915536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.915928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.916870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=7ea7d2d4fd034e66aa589875e3e27ce8) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:57.917536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:57.919015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:57.920605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97ced7-9b52-47cd-bb5f-52922da39a91 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:57.933452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 273/539] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/role_assignments?user.id=7ea7d2d4fd034e66aa589875e3e27ce8&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:57.933452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:57.933452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:57.933452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:57.962680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6abaed-d35a-425d-979b-812b58861d2b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:57.969429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:57.970644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:58.047794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:58.049133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:58.049721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:58.050245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:58.066521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:58.068295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:58.077875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:58.093749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 274/540] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:17:58.103396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:58.104339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:17:58.105209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:17:58.108055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 267/541] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:17:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:58.113123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:58.114127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:58.242023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5300b9-c63b-4903-ac6b-ef571805b12e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:58.285133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:58.285975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:58.287963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:58.288241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:58.304051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ff9fc-7634-44cf-9f50-2b768fddd492 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:58.317231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:17:58.325099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:58.326077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:17:58.326888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:58.327510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 275/542] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:58.333722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:58.401804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:58.402176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:58.402911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:58.403202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:58.403581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:58.404418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:58.404735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:58.406397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:58.429451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 268/543] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:58.443607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:58.449711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:58.461210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67682bc-00d0-4d80-bf04-27e27a78c21f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:58.530678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:58.531693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:58.532155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:58.532601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:58.533643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:58.534121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:58.535724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:58.560779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 276/544] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:17:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:58.574036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40dab7-dade-4ec5-b1c1-719789022076 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:58.577000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:58.619883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:58.620538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:58.620815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:58.621085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:58.622015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=84609c4a913144e2a71c508e96c53d0a) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:58.622434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:58.623823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:58.632118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 269/545] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:17:58 2022] GET /identity/v3/role_assignments?user.id=84609c4a913144e2a71c508e96c53d0a&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:58.743947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef83fb-a10b-4e6c-b288-6fec66b9372d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:58.760633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:58.793161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:58.794197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:58.794405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:58.794715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:58.808027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:58.809723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:17:58.836975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:58.837902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:58.838790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:17:58.838790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:17:58.838790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:17:58.838790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:58.841933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 277/546] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:58.847561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:58.950792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:59.003610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4be31-0f1a-404d-89db-bfd0fa0b6b6f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:59.052856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:59.053564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:59.053846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:59.054120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:59.054848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:59.055150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:59.056519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:59.072707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 270/547] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:59.078219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:59.212209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c3508-109c-4f82-909b-9f67a10719fe None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:59.252593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:59.253400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:59.253705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:59.253984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:59.266381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:59.267975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:17:59.288283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:17:59.290095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:17:59.291193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 278/548] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:17:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:17:59.295715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:59.456790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849cac0-18cf-4ce9-9fdb-413e68ce0bb2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:59.495437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:59.511037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:59.512060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:59.512547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:59.513739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:59.514849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:17:59.515366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:59.517026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e4abf8-651b-47cf-ab5f-297e1f64cd59 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:59.536173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 271/549] 10.222.0.24 () {56 vars in 1135 bytes} [Thu May 19 13:17:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:17:59.541849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:59.544935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cfef0-3f34-4f04-b447-bce51caaef10 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:59.545542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cfef0-3f34-4f04-b447-bce51caaef10 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:59.546044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cfef0-3f34-4f04-b447-bce51caaef10 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:59.547840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 272/550] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:17:59.553190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:59.553774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:59.554280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:59.581821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:59.584437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:17:59.585887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:17:59.696874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aebf76-4a76-4476-ae77-7c08bc7ba283 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:59.707535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 273/551] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:17:59.742162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:17:59.742810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:17:59.743090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:17:59.743367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:17:59.757021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:17:59.857429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:17:59.858878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:17:59.912094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:17:59.942640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcfa62-a1bb-42f9-93a1-f4ebf77012b3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:17:59.970795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.014818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.015537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.015821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.016098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.032317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:00.048208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 279/552] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:00.049833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:00.050867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:00.051729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:00.052394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.060633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 274/553] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:17:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:00.067412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.182990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b6e14-7438-42bf-8bb5-509b568b8ca3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.231043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.231791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.232096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.232310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.233074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:00.233383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.234806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:00.256537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 275/554] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:00.262550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.294629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7940a-177b-4457-a92c-b5fd60cee7b2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.332421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.343133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.344280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.344774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.345259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.357783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.360598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.361651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aceb0675-45c3-4115-afcc-04f1f2c7bb13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:00.384306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:00.385273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:00.386249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:00.386249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:00.386249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:00.386249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:00.388777 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 280/555] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:00.399770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a5dd-bf75-471b-b5c6-01bc1a59db5e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.401034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a5dd-bf75-471b-b5c6-01bc1a59db5e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.401560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907a5dd-bf75-471b-b5c6-01bc1a59db5e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.404178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 281/556] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:00.405130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a5dd-bf75-471b-b5c6-01bc1a59db5e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.456420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7686f60-e670-47dd-b723-4d0bc82d6ace None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.491386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.492206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.492486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.492756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.504286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.505774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-bcfe3bbe-5b59-4f2f-aaec-e3bda7c47623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:00.523513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:00.524523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:00.526040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:00.526040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:00.526040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:00.526040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:00.526040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 276/557] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:00.526347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.526878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.526878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.545236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907a5dd-bf75-471b-b5c6-01bc1a59db5e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.545583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:00.546735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:00.595423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.596165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.596444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.596713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.597516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:00.597820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.599253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:00.616462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 282/558] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:00.617381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.641184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 277/559] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:00.642173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.661226 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.670688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.738726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a4f9-c2b5-4ae9-addc-7c5f194f8e12 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.749207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c23cf-bd7c-4cbd-bc6f-c5395e6699cd None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.775276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.776351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.776846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.777440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.778568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:00.778882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.780661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:00.786116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.786937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.787217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.787486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.794429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 283/560] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:00.794579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.795863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.796064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-78f8b095-9bb6-45e5-977f-dc7bcde655ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:00.812924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:00.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:00.814878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 278/561] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:00.815988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e0bf-08b4-4cde-a3e5-b07a04cf45e2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.815988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e0bf-08b4-4cde-a3e5-b07a04cf45e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.815988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07e0bf-08b4-4cde-a3e5-b07a04cf45e2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.821445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 279/562] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:00.821445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e0bf-08b4-4cde-a3e5-b07a04cf45e2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.908494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.912756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885b80-377d-44b5-a1c6-d0d435a4affb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.913052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:00.938637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07e0bf-08b4-4cde-a3e5-b07a04cf45e2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:00.948504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.949306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.949624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.949901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.961265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:00.969237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:00.970433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 284/563] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:00.976063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:00.980048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:00.980858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:00.981143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:00.981457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:00.990590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:00.992082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-30dd42a9-e19f-45f2-b6a6-906af9bcbbb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:01.007074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:01.008084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:01.009073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:01.013136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 280/564] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:01.014738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.015036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.015319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.033750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:01.034956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:01.081512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c105d-8747-41d0-b2a7-ac13494af985 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.103357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 281/565] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:01.104203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.121044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.121790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.122097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.122377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.123228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:01.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.125008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.137639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 285/566] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:01.138457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.157707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.169331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.206224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f974c-1588-40ae-83f9-116030ac455f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.239656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.240319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.240559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.240827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.241610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:01.241913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.243275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942efb2b-ce86-4904-add8-6801390409ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.243515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e485b-69a5-415b-bb34-473e785d6e64 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.251380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 282/567] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:01.252615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48651a-35e2-4dfa-a552-8a07599fdf47 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.252897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48651a-35e2-4dfa-a552-8a07599fdf47 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.253164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48651a-35e2-4dfa-a552-8a07599fdf47 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.254406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 283/568] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:01.255083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48651a-35e2-4dfa-a552-8a07599fdf47 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.285502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.286170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.286459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.286735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.287571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:01.287879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.289669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.306484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 286/569] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:01.307292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.358855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48651a-35e2-4dfa-a552-8a07599fdf47 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.381797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.386852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.393729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.394528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.394802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.395070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.404428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cdc8f-51e9-4c1d-ac30-6ba067fdbd5c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.405041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:01.412568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:01.414050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 284/570] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:01.420214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.447809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.448460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.448744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.449019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.449887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=78b2f9c966e24c46a23631b80a578862) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:01.450230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.451665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c5a63-4bb2-472b-a95c-54d8a7e84ef7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.457442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 287/571] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/role_assignments?user.id=78b2f9c966e24c46a23631b80a578862&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:01.458603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.458889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.459166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.478197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:01.479447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:01.528820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1215100-0a05-4c04-939f-25f12071bbec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.555031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 288/572] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:01.555816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.581610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.582423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.582713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.582958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.595115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.596603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:01.614773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:01.615728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:01.621898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:01.621898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:01.621898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:01.621898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:01.621898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 285/573] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:01.621898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.641032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.668820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.676905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f643b-637e-4da2-bdca-669e164a2e86 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.714224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.715027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.715309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.715587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.726128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.727593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.730864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7204-a6cc-407e-baed-0975bba0abba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:01.746019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:01.746980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:01.748053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:01.748053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:01.748053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:01.748053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:01.748053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 289/574] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:01.748627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.779325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.779983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.780278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.780566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.781317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:01.781653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.783033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.797486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 286/575] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:01.797486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.858782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d667a6a-8aa5-4cbe-895c-6eade77fe600 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.883173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.896474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.897272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.897586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.897863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.908999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:01.911951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4423f8-0314-40e4-9498-adda2445ac43 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:01.915813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:01.917487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:01.918969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 290/576] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:01.920069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:01.957358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:01.958051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:01.958337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:01.958616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:01.960276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:01.960601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:01.962064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:01.980544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 287/577] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:01.980544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.031614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309e5fc-1793-41e3-bc9b-db850cb18b98 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.066699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.067371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.067660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.067940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.068675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:02.068974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.070400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.082983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 291/578] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:02.083682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.096298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150cfbd-508b-48f7-8164-e692e77b0423 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.128427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.135178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.136207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.136645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.136918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.147954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.149956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:02.168169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:02.169225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:02.170181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:02.170181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:02.170181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:02.170181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:02.170181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 288/579] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:02.170487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.192148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.192457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69318-36a5-4630-9644-55c7afc4a974 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.227851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.228882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.229169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.229460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.230217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:02.230524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.232147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.244808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 292/580] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:02.244808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.298746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa8d2-091e-4499-b105-ec296e49b900 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.336089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.336895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.337218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.337520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.345081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.346562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.348601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b144331-7c38-431a-8bb3-b5516ab02e23 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.351719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:02.361538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:02.362734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 289/581] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:02.364011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.393531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.394265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.394548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.394808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.402134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.403633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:02.418572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:02.419594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 293/582] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:02.425897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.449041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.474481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acede9-3dfb-4dbb-9e38-65b260c764ab None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.508517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.509171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.509473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.509753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.510500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:02.510802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.512164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.525453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 290/583] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:02.525453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.532352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde8bb2-a3ee-4e6d-9ed4-c301d004f620 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.569391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.570214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.570499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.570780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.582232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.583735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:02.609344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:02.610492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:02.611603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:02.611603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:02.611603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:02.611603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:02.615345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 294/584] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:02.616209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.634782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35e6e-70d5-48d6-8b71-dd9311c4ba86 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.635374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.674764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.675355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.675635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.675908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.676752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:02.677060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.678544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.682477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.700243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 291/585] 10.222.0.24 () {56 vars in 1137 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:02.701299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.729846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d51fc-a48f-4b28-b1df-161c4a14014a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.767504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.768168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.768449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.768733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.769545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:02.769859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.771383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.785445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 295/586] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:02.785445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.813758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2d910-6cda-405d-9d4a-6522a91950dc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.849165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.849986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.850269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.850541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.858383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.859825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.864083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:02.874594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:02.875575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 13:18:02.876545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:02.880671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 292/587] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:02.881444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.891039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75721f4-f438-400d-9634-d6b6191ea468 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:02.926666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:02.927258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:02.927846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:02.928127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:02.929006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:02.929317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:02.930749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:02.940236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:02.948534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 296/588] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:02.948534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:02.993168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8cf77-8a0f-42a1-b59f-e7af24eca094 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.035206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.036004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.036439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.036720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.050588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.052030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df364ff2-c569-4d34-9851-43bb701722ad None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.052160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:03.074451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:03.075414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:03.076419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:03.076419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:03.076419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:03.076419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:03.077184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 293/589] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:03.077538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.086573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da797-92f2-4504-808d-29b985454dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.087460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da797-92f2-4504-808d-29b985454dea None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.087739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da797-92f2-4504-808d-29b985454dea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.088013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da797-92f2-4504-808d-29b985454dea None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/862f9782f8574b8bb99d38c51333277f/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.092178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.105539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33da797-92f2-4504-808d-29b985454dea None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=862f9782f8574b8bb99d38c51333277f, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.107008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33da797-92f2-4504-808d-29b985454dea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.121809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 297/590] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:02 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/862f9782f8574b8bb99d38c51333277f/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:18:03.123058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da797-92f2-4504-808d-29b985454dea None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.210696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc407-9c6a-457a-b34c-dfb380ab9c93 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.220560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.255988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.256964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.257462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.257990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.259088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:03.259555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.261357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.276811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da797-92f2-4504-808d-29b985454dea None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.284716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 294/591] 10.222.0.24 () {56 vars in 1137 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:03.285925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.335579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.336389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.336669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.336946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.345586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.346686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:03.361761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:03.363363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 298/592] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:03.364561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.408118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d89f65-0431-479f-98cb-26b5d69168be None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.415630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.448426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.449183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.449567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.449925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.450870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:03.451245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.452799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.465655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 295/593] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:03.466507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.474522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b956af-811e-4f71-adcc-d40ada06d949 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.494087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.510155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.510970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.511251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.511525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.523295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.524926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:03.544642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:03.552333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:03.554545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:03.554545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:03.554545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:03.554545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:03.554545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 299/594] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:03.554545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.583231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464ba1-5725-48bc-bff1-4aa63f2e4aa2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.619681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.620602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.620885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.622081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.623348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:03.623666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.625226 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.648573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.650786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 296/595] 10.222.0.24 () {56 vars in 1139 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:03.651717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.657101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3462d40-6aac-4bd6-b7c9-a0ced4d6654c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.693806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.694605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.694882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.695153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.702801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.704260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:03.718807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:03.719888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:03.720971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:03.724655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 300/596] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:03.725348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.748063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.767085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b47449-f310-421a-a61a-4d24a1c18646 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.802047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.802705 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.802985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.803257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.803969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:03.804269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.805658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81549a86-5c04-4698-8ef8-656500be5004 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.821711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 297/597] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:03.821711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbedd2-c7cc-4a48-b240-d98cc1d31775 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.821711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbedd2-c7cc-4a48-b240-d98cc1d31775 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.821711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbedd2-c7cc-4a48-b240-d98cc1d31775 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.824129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 298/598] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:03.824925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbedd2-c7cc-4a48-b240-d98cc1d31775 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.835993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ccbe3-cf0b-4756-a0aa-e1f2a270420d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.873805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.874619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.874878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.875149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.886014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.887463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:03.905314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:03.906505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:03.907715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:03.907715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:03.907715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:03.907715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:03.907715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 301/599] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:03.907715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:03.922457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:03.936038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abbedd2-c7cc-4a48-b240-d98cc1d31775 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:03.971614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:03.972192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.972540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.972804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.973541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:03.973856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:03.975260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7da3c5-5abc-4830-bee3-6b0bb1f43925 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:03.988671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 299/600] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:03.989935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:03.990215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:03.990485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:03.995809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.009308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:04.010534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:04.019636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e09d2-0f48-4982-aac9-ef96ed4d9ce2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.053208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.054160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.054539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.054823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.061641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.063097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:04.076968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:04.078078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:04.079187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:04.085464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 302/601] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:04.085464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.085665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 300/602] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:04.086749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.160536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.192488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8db11-473b-474d-908b-2fc1e6c14e70 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.194724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e78cc5-80a2-4697-84a9-71029dc2c5fc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.225788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.226763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.227647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.228161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.228536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.229387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:04.229866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.231278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.231401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.232262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.232539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.232810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/d92108b779ca47aca8f941b7dfae80d5/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.245923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 303/603] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:04.246922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.252531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=d92108b779ca47aca8f941b7dfae80d5, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.254032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.272514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 301/604] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:04 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/d92108b779ca47aca8f941b7dfae80d5/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:18:04.273580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.382811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abc3a4-748b-49d0-a645-5ee2e13877b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.407465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b83b-8f5f-49b1-a49d-a90a79f5883b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.415989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.424109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.425131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.425620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.426072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.427126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:04.427632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.429302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.444242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 304/605] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:04.445555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.446039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.446843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.447177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.447446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.459511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:04.467938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:04.469002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:04.469896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:04.474059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 302/606] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:04.474979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.522383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.566981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522889d-2fb1-4a17-b890-626296287b67 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.591960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400498d-bd82-4258-b0b9-e7de876d3b9c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.609021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.610157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.610506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.610788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.620198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.622280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.627384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.628427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.628801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.629167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/7ea7d2d4fd034e66aa589875e3e27ce8/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:04.639037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:04.640181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:04.641321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:04.645324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 305/607] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:04.646565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.649530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=7ea7d2d4fd034e66aa589875e3e27ce8, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.651066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.668576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 303/608] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:04 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/7ea7d2d4fd034e66aa589875e3e27ce8/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:18:04.669957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.749785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.861138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737d16d-c43b-411c-902f-8fb037ef52b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.869798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:04.884999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cf5b6-087b-408f-8a9d-aa744847ad19 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:04.924680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.925629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.925915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.926190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.927166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:04.927799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.929508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.936797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:04.937594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:04.937949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:04.938296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:04.939166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:04.939572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:04.939695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 306/609] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:04.941161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:04.946394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:04.972569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 304/610] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:04.974107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.108473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24dce7-ee16-4be7-9ce4-fe0b75b8e722 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.136591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:05.155392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915dfeb-c9ec-458b-a049-3482c50be1c0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.188101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.188917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.189205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.189509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.192241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.193142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.193439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.193842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/84609c4a913144e2a71c508e96c53d0a/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.210251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=84609c4a913144e2a71c508e96c53d0a, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:05.211836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:05.218684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:05.220193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:05.234218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 305/611] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:04 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/84609c4a913144e2a71c508e96c53d0a/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:05.266460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:05.267462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:05.268598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:05.268598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:05.268598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:05.268598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:05.268598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 307/612] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:05.273962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.277509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:05.397167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb75f0-976f-4ec9-88ae-7503dc9cd65f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.436612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.437777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.438293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.438801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.439748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:05.440270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:05.441909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dd797-a6b0-48c4-8d44-2346c5853970 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:05.455597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 306/613] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:05.460636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.557855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd934-1a30-4b5b-aaf1-0fb714bd0550 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.558509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd934-1a30-4b5b-aaf1-0fb714bd0550 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.558988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfd934-1a30-4b5b-aaf1-0fb714bd0550 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.564912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 307/614] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:05.566647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.567148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.567623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.571502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cf8d1-7cf2-4dd0-becb-d2aa71326c7b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.586354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:05.587941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:05.609713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.610518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.610806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.611087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.619565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:05.621060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:05.636721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:05.637904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:05.639715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 308/615] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:05.644474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.655717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:05.673392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:05.673636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 308/616] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:18:05.694067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.757987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d939ff-174c-4371-b444-e185b36dfb26 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.795723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.796689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.797167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.797656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.798706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:05.799201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:05.800850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:05.806165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27541d6e-3859-4659-9854-819cb6b88d8e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.819858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 309/617] 10.222.0.24 () {56 vars in 1135 bytes} [Thu May 19 13:18:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:05.825558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.844372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.845234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.845556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.845830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.857650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:05.865690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:05.866650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:05.867524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:05.870748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 309/618] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:05.873217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:05.919377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:05.939995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf9289-381b-4af5-9f6e-30f051c066f9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:05.948764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:05.977864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:05.979026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:05.979516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:05.980024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:05.991793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:05.993766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:06.014067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:06.015082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:06.016019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:06.016019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:06.016019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:06.016019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:06.018380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 310/619] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:06.023997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.044312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd0e7-bc20-47f7-97fb-8c6e17b4302b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.135241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca35e63-6939-439e-8972-f5e052df1087 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.142671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.143353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.143637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.143909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.144659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:06.144965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.146358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.165150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 310/620] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:06.171458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.175577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.176702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.177195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.177704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.178643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:06.179168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.180757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.189812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:06.199629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 311/621] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:06.205396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.365704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cf357-b116-481a-933f-9cbd33bc5573 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.399157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254767e5-e037-404e-84ee-20a394f9acbe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.426732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:06.442145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.443023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.443238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.443521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.453647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.454642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.455071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.455487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.461792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.463286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.467558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.469160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:06.487079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:06.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:06.489522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:06.489522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:06.489522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:06.489522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:06.489864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:06.490910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:06.492560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 312/622] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:06.494282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.499592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 311/623] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:06.501134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.576369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:06.609534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d1d6-ebe4-423b-af57-bd310ea6cfd7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.646146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.647544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.648025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.648476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.649570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:06.650154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.651855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.670422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 312/624] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:06.671956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da28476-69e4-4fd3-83f8-62632591a9cc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.672307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.701225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:06.770261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.770960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.771255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.771572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.772396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:06.773067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.774239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.782722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83420dd-1932-4c52-9d02-2c6af7eef894 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:06.804006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 313/625] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:06.814400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.819830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:06.820929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:06.821429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:06.821886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:06.830518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:06.832507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:06.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:06.849473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:06.850484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:06.857566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 313/626] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:06.859062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:06.997269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965b8b-0957-4b78-a107-6c6e053deca1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.016146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.041883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.063724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.064879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.065844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.066138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/78b2f9c966e24c46a23631b80a578862/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.070876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae02cd-bc45-4619-9941-ebf10fbd281f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.106892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=78b2f9c966e24c46a23631b80a578862, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:07.108450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.108590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:07.109487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.109991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.110472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.111639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:07.112164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:07.113809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24876c-4a78-4b2e-b1f0-0ff4fcd30a19 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:07.133210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 314/627] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:06 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/78b2f9c966e24c46a23631b80a578862/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:18:07.149777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 314/628] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:18:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:07.156146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.167654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5811-1e56-4dc9-a1c6-21085188e751 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.167654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5811-1e56-4dc9-a1c6-21085188e751 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.167654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5811-1e56-4dc9-a1c6-21085188e751 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.169191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 315/629] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:07.181648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.181949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.182208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.224049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:07.226794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:07.322517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 316/630] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:18:07.353455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.354660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5dd4f-34f9-487a-a8ca-c881aa667931 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.391837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc154f5-654c-4237-941e-33ae054d04be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.392491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc154f5-654c-4237-941e-33ae054d04be None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.392772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc154f5-654c-4237-941e-33ae054d04be None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.393047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc154f5-654c-4237-941e-33ae054d04be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.428125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.453528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.480280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc154f5-654c-4237-941e-33ae054d04be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:07.482112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc154f5-654c-4237-941e-33ae054d04be None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:07.499185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff363c72-77fe-4b00-9789-15dace013f9f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.546449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3189656-c3a2-4b38-b9d2-1a04686a1a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.547507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189656-c3a2-4b38-b9d2-1a04686a1a20 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.548158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189656-c3a2-4b38-b9d2-1a04686a1a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.548579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189656-c3a2-4b38-b9d2-1a04686a1a20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.563563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3189656-c3a2-4b38-b9d2-1a04686a1a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f3189656-c3a2-4b38-b9d2-1a04686a1a20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:07.574139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:07.575331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:07.576429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:07.578967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 317/631] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:07.585580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704bca1-83e7-4ac4-a58e-73e7a4569f65 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.585580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704bca1-83e7-4ac4-a58e-73e7a4569f65 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.585580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704bca1-83e7-4ac4-a58e-73e7a4569f65 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.585580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 318/632] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:07.586139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704bca1-83e7-4ac4-a58e-73e7a4569f65 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.599153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 315/633] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:07.600916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.601269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.601711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.619225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:07.620663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:07.683983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.693923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 316/634] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:07.695076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.701747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704bca1-83e7-4ac4-a58e-73e7a4569f65 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.738253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.738736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.739019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.739290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.740044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:07.740347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:07.741886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:07.743292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.753385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 319/635] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:07.754177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.807438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8dae5-cc31-4924-a939-7e80cad78f05 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.851271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.852080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.852363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.852638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.863455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:07.864965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:07.872956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125f563-1167-44bf-bb69-7f0c9d6a15d1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:07.883669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:07.884607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:07.885561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:07.885561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:07.885561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:07.885561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:07.889486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 317/636] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:07.890346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.918046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:07.919592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:07.920435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:07.921136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:07.934873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:07.939647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:07.947445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:07.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:07.949336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 320/637] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:07.950416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:07.996202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b7e46-9337-419f-9ae8-b7af5f67414d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.013264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.032606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.033334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.033644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.033920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.045337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.047333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.059102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99791e4-49a3-4b6a-9e7e-8d15eaa24837 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b7cbf6b9-b917-496c-97cc-762d75c99ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:08.065576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:08.066584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:08.067514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:08.067514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:08.067514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:08.067514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:08.067514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 318/638] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:08.068698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fb8c6-726a-4b85-8d8b-16515b6f4d2f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.068698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fb8c6-726a-4b85-8d8b-16515b6f4d2f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.068698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fb8c6-726a-4b85-8d8b-16515b6f4d2f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.073761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 319/639] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:08.074572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fb8c6-726a-4b85-8d8b-16515b6f4d2f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.105635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.106304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.106612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.106889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.107803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:08.108113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.109505 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.125839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 321/640] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:08.125839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.186906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.192549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fb8c6-726a-4b85-8d8b-16515b6f4d2f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.227785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.228206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3ddf3-c7fd-4799-9ae6-e536fbf8ef11 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.228677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.228761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.228978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.229760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:08.230067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.231480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b233e7-8613-407e-957b-d341792b6bae None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.240941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 320/641] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:08.241187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.241663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.241769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.251146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.259414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:08.261009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:08.275189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.275860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.276170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.276406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.277124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:08.277442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.278859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.290679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 322/642] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:08.295627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.330861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 321/643] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:08.342799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.401939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d044c7-ae0c-4498-838e-a7d4fb21f602 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.442107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.443079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.443193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.443441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.445990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.451092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.452281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.457435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228397e5-d9bc-4284-958b-3eea23c984f3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:08.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:08.470684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 323/644] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:08.476727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.495863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.497914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.498268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.498613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.509696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.511096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.514955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:08.529899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:08.531045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:08.532317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:08.532317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:08.532317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:08.532317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:08.532728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 322/645] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:08.533530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.593320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0cfa8-d7f0-45fe-86c5-a679afe44821 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.628886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.629710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.629989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.630259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.638721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.640223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.641326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f56371-2939-4294-8f84-fc928c63de60 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:08.655220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:08.656221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:18:08.657194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:08.661140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 324/646] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:08.661753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.687651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.688449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.688880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.688988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.699896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:08.707670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:08.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 323/647] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:08.710158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.712345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.767948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.774274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b57f9-3a2e-4860-8474-673689820361 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.811844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.812798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.813269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.813633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.814487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:08.814860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.816346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.818141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea2d6d-c1de-49b6-860b-2c3a4a331eb3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.832365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 325/648] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:08.832365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.858904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.859642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.859912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.860200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.860950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:08.861258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.862890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.875314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 324/649] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:08.876130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:08.942524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438775d-8c2a-4b3d-af8d-6aa7a0474e41 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.946955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:08.976514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:08.977190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:08.977503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:08.977781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:08.978630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:08.978941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:08.980373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:08.990690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b34e3d-a607-482a-9cdf-34a9d61a350e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:08.998755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 326/650] 10.222.0.24 () {56 vars in 1137 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:09.003780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.019340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.027002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.027675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.027957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.028278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.029067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:09.029341 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.030759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.039692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 325/651] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:09.040318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.112572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e238be-0670-4d16-a459-b1828b8876ec None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.146114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb825579-5c2c-443a-a541-a794f8385e00 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.150602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.151266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.151549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.151822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.152669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=862f9782f8574b8bb99d38c51333277f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:09.152980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.155117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.173470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 327/652] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/role_assignments?user.id=862f9782f8574b8bb99d38c51333277f&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:09.173470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.174440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.188728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.189602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.189889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.190165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.199683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.201398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:09.217897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:09.219132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 326/653] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:09.220323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.288053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.326295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eafc1-9c49-4415-9efa-96556bfd3861 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.354775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299aa55-c307-45de-ad0c-00e8bcb46a5f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.380728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.381772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.382913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.383393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.406160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.406374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.406785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.406922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.419987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.421512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:09.441021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:09.442032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:09.442982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:09.442982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:09.442982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:09.442982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:09.442982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 327/654] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:09.443272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.479350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.480974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.487653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.563989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35738312-4595-40ee-9da0-e589ffd4ba81 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.582347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.600041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.600720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.601006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.601285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.602259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:09.602671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.604123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbad7c-07a5-4fa4-ac11-73a751c93518 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.610769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 328/655] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:09.610769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.631922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 328/656] 10.222.0.24 () {56 vars in 1137 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:09.631922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ab83-e8f8-46e2-9dda-10fd5eb7b5b6 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.631922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ab83-e8f8-46e2-9dda-10fd5eb7b5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.631922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364ab83-e8f8-46e2-9dda-10fd5eb7b5b6 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.631922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 329/657] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:09.631922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ab83-e8f8-46e2-9dda-10fd5eb7b5b6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.726187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fed2d6-5c11-488d-ac22-67032407eb0f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.730537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.739097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364ab83-e8f8-46e2-9dda-10fd5eb7b5b6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.763415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.764086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.764365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.764642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.765368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:09.765703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.767061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.780313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.780930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 329/658] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:09.781837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.785509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.785795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.786075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.796910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:09.798390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a1830424-bf1e-4a68-9f21-40254e183a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:09.816356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:09.817317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:09.818320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:09.818320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:09.818320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:09.818320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:09.818320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 330/659] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:09.818907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.819108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.819382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:09.831856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:09.838081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:09.839291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:09.896591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc096f-5785-44c3-a872-d1cf03db0f9f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:09.909609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 331/660] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:09.910676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:09.935437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:09.936097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:09.936489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:09.936763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.004529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.006053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.011009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.036503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e6645-9005-46d7-89a8-ee6c8e8bceb5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.053949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.077993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.078800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.079084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.079360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.086666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.088114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:10.103897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:10.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 13:18:10.105962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:10.110063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 332/661] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:10.110944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.129646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 330/662] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:10.134763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.219146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ae0a3-0aff-46e6-b7d7-a520d0beb779 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.235553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3868f2-f4cc-4069-b58c-98c84efd4f78 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.259183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.260138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.260599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.261334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.262198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:10.262498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.264100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.267439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.272998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.273813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.274094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.274367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.278209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 333/663] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:10.279280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.285201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.289438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:10.298085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:10.299152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:10.300058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:10.304351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 331/664] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:10.305331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.417717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc9550-6edc-4f36-b4ed-77c7fa56ae8d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.455668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.456457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.456737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.457007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.470513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.473018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:10.494517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:10.495685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:10.496654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:10.496654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:10.496654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:10.496654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:10.496654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 332/665] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:10.496654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.530941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ab1df-57f3-4b66-9c1e-d93ce2cf71ef None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.557613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.571146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.571942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.572318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.572690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.573684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:10.574095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.575743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.594057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 334/666] 10.222.0.24 () {56 vars in 1139 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:10.594879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.618791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122f39f-0e5e-41d9-a095-bd9df7375018 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.652959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.654430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.655688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.656022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.656429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.665955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.667401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:10.681959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:10.682916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:10.683880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 333/667] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:10.684396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.711246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f19d7-52c4-4541-a4ab-d7449c3c2dbe None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.747034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.747940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.748344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.748637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.749385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:10.749721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.751096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.759741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 335/668] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:10.764827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.789231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe07ddc-3857-4fd1-a6a5-16ad4c39a3c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.801022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.828364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.829097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.829377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.829687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.830412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:10.831150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.832841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.849754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 334/669] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:10.851280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:10.869562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f08a-0245-48a1-be9d-693ee0c152ec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.888487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:10.907990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.908590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.908871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.909146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.958397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ce8b6-134d-4de8-93c0-cffce5bbf765 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:10.981757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.983232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:10.994837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:10.995808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:10.996204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:10.996448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:10.997377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:10.997672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:10.999222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.013666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 335/670] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:11.014487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.048631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.102555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 336/671] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:11.103336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.121573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.124852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5c2b7-8969-462c-a773-d2c5e7913c24 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.162172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.162972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.163421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.164082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.174682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.176218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:11.194680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:11.195657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:11.196619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:11.196619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:11.196619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:11.196619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:11.196619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 336/672] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:11.196987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.223290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb310e-6984-46da-b5fc-b4942b6c4ebc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.266965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.267770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.268050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.268322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.278272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.281077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.286746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:11.299513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:11.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:11.302188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:11.305376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 337/673] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:11.306210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.307900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea0a08-16fe-494f-80a4-f2869b8a59fd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.344023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.344843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.345902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.346200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.347050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=d92108b779ca47aca8f941b7dfae80d5) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:11.347370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.348819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.357607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 337/674] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/role_assignments?user.id=d92108b779ca47aca8f941b7dfae80d5&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:11.358455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.409973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.447696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aab77-fa99-4b24-80e6-ff70e429d7e9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.502404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.503225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.503512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.503790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.508697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435baeed-fae7-412d-9a17-a2b41193e0c3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.522133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.524843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:11.550144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:11.551326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:11.552461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:11.552461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:11.552461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:11.552461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:11.552461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 338/675] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:11.552816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.559378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.560373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.560697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.560974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.561798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:11.562115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.563609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.579003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 338/676] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:11.579854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.589016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.668699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b6e32-abda-48ba-940a-f7d1e217354c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.690652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956851d4-a475-4e07-b60e-802bf0686062 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.694981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.704339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.705003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.705283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.705603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.706395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:11.706707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.708238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65362e28-4e68-45e7-8dca-a83aaaad9c9a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.720974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 339/677] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:11.722106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329627-fac3-4879-91db-2447a3e33a60 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.722395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329627-fac3-4879-91db-2447a3e33a60 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.722673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329627-fac3-4879-91db-2447a3e33a60 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.723875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 340/678] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:11.724559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7329627-fac3-4879-91db-2447a3e33a60 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.730769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.731628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.732506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.732793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.733568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:11.734315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.736041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.753492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 339/679] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:11.753492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.823681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:11.834727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7329627-fac3-4879-91db-2447a3e33a60 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.861057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3082e10-67c9-44a4-b07a-3c1ec0d614ad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:11.874999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.876261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.876678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.876941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.885024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.886502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.897687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:11.898349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.898632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.898896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.899729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=7ea7d2d4fd034e66aa589875e3e27ce8) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:11.900039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:11.901572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-48eb3423-eda4-40a2-8741-b9d1aee0e2d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:11.902038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:11.902988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 341/680] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:11.909102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:11.909881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:11.909951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 340/681] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/role_assignments?user.id=7ea7d2d4fd034e66aa589875e3e27ce8&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:11.909951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:11.931664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:11.932888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:11.983912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.013109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 342/682] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:12.014237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.019411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafbb1f-b01a-4686-893f-69e0b83158e3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.048389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.054664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.055868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.056244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.056715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.064251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.065945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:12.081463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:12.082435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:12.083371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 341/683] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:12.083907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.136191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd786fd-c2f1-4dde-95c2-095798fd92c0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.179441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.180123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.180404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.180680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.181556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:12.181866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.183351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.188327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a21f59-21ee-415d-bb18-977cc4364e05 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.203619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 343/684] 10.222.0.24 () {56 vars in 1135 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:12.204480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.225256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.226880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.227795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.227795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.244725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:12.252239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:12.253751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:12.254722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:12.255593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:12.260360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 342/685] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:12.261546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.321461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.341202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e88fa3-5c04-40f2-8d04-233ed015c79c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.378651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2e72-63db-4800-8649-f9d5065fa01d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.389687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.391222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.391684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.392138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.393372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:12.394346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.396462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.413295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 344/686] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:12.414806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.415457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.416283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.416636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.416974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.490024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.491528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.547011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.549077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.557353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d383f-6dfc-4a76-914e-d1de0ecee61e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.604022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 343/687] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:12.604830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.609102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.610066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.610526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.611017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.611992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:12.612504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.614634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.627193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 345/688] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:12.628600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.734885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f09ad-9f3f-4f0a-9bc0-2ad87b7116da None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.745388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeed28e-a07b-400e-aec7-8f93da4bd0e0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:12.776994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.777739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.778047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.778359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.779278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=84609c4a913144e2a71c508e96c53d0a) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:12.779594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.781039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.783363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:12.784810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:12.785534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:12.785986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:12.787649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 344/689] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/role_assignments?user.id=84609c4a913144e2a71c508e96c53d0a&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:12.789089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.800043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:12.802856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:12.806265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:12.826056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:12.827274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:12.828193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:12.828193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:12.828193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:12.828193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:12.831387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 346/690] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:12.840464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:12.866657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:12.976318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43d00f-a62f-44fd-b80f-bbdfca46918c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.031427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.032225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.032506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.032872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.039944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c55ee-4e40-4d9d-9a86-e6af12cf5bc5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.044246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.045937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:13.065002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:13.065980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:13.066977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:13.066977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:13.066977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:13.066977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:13.066977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 345/691] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:13.075227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.075412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.076116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.076505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.076787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.077553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:13.077878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.079540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.094573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 347/692] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:13.099545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.151869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:13.190821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08844b1-b7dd-45d0-84a7-9bf1d9d4d1e2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.197237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:13.231001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.231735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.232016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.232291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.233128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:13.233472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.234962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.236933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337dd9b-a8bd-4f42-8890-0dbba7784ecb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.252214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 346/693] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:13.257479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.291169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.292237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.292697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.293114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.301695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.303503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:13.343274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:13.344302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:13.345224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:13.347192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 348/694] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:13.352744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.380234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97b4d4-4496-4cf0-bba5-3a231250a9da None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.409512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:13.422053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.423351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.423840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.424338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.434015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.436433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:13.452984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:13.454065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:13.455060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:13.458006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 347/695] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:13.472913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.521548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ded51-d982-493f-bfd8-9a55df35f6f1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.551709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:13.558131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.558800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.559087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.559388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.560255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:13.560569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.565226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.579380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 349/696] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:13.591095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.599931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a1cf8-3747-4308-9be5-c9e9a39b5865 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.643741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.644833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.645323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.645827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.647002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:13.647522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.649323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.684753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 348/697] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:13.690212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:13.723455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204513f0-ed50-47f7-9f77-4b6f34cb4e28 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.760993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:13.769553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.770732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.771015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.771291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.772400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=78b2f9c966e24c46a23631b80a578862) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:13.772713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:13.775099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea363e7-3244-4d84-a01b-59e9b5df4c96 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:13.783416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 350/698] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/role_assignments?user.id=78b2f9c966e24c46a23631b80a578862&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:13.785610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077996-5cb4-4a7e-bf6a-8a97e0bc80d5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.785900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077996-5cb4-4a7e-bf6a-8a97e0bc80d5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.786174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077996-5cb4-4a7e-bf6a-8a97e0bc80d5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.788060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 351/699] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:13.795762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.796034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.796366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.815212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:13.816559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:13.831789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd30f2-0cd4-4fb4-af40-e79a91104af0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:13.876181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:13.881965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:13.882915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:13.883199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:13.883470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:13.913932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 352/700] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:18:13.980117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.018012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.019514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.072998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:14.122535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1d402-9d81-465f-90f9-1ea2ce57b50f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:14.140106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 349/701] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 13:18:14.145835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.167655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:14.168474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.168783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.169069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.170223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.171909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:14.189935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:14.191005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:14.191979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:14.194648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 353/702] 10.222.0.24 () {56 vars in 1129 bytes} [Thu May 19 13:18:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:14.198156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.266878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:14.271769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a09946-9f86-4453-81b3-9aee4134024f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:14.314193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:14.315243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.315675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.316099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.330102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.331530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455c6b9-70f7-4041-b1f1-c8443e5af3ea None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:14.331958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.335787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:14.354408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:14.355396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:14.356367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:14.356367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:14.356367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:14.356367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:14.357022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 350/703] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:14.363003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.374952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:14.376041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.376521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.377108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.378080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:14.378585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.380279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.385757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 354/704] 10.222.0.24 () {56 vars in 1124 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:14.391999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.522295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a84a1b-6b9b-4159-a13a-cea175acf136 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:14.538932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:14.554424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e622dfa-908e-423b-9e4b-c8d5bfe5b8b8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:14.566041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:14.567072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.567560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.568495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.569463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:14.570032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.571674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4264ac-a504-48a6-87cf-b29a6248540a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.585776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 355/705] 10.222.0.24 () {56 vars in 1124 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:14.637929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c447ccb-13fd-4844-acc0-5e9b243c131e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.638302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c447ccb-13fd-4844-acc0-5e9b243c131e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.638501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c447ccb-13fd-4844-acc0-5e9b243c131e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.640106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 356/706] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:14.645820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.646201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.646368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.647797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:14.648478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.648761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.649040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.649803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:14.650113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.651785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.670457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 351/707] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:14.677039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.682680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:14.689112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:14.787340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:14.831827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 357/708] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:18:14.855460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb79f71-35c7-4599-8612-64bd841b2ba3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:14.863230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:14.899929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:14.901879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:14.902425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:14.902891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:14.912669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:14.916167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:14.935461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:14.936483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:14.937473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 352/709] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:14.949679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:15.009698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251e935-5b14-4444-bbc9-ea2600629a04 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:15.018122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:15.050436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:15.051953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.052472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.052938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.065293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:15.067106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:15.087359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:15.088334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:15.089239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:15.089239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:15.089239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:15.089239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:15.092251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 358/710] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:15.098614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:15.202801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5b07-3636-41cb-91da-c4108e576446 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:15.214547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4ed18-ce42-469f-916b-443ebfcc19c2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:15.239514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:15.255495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:15.256884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.257367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.257872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.258793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:15.259289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:15.261575 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:15.275170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 359/711] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:15.280975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:15.298113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:15.299055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.299055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.299345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.300091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:15.300395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:15.301899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:15.303593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:15.331954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 353/712] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:15.337449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:15.397170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93223dee-19a2-4669-bbdf-2bc67a97a6cd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:15.439620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f5e23-6fc1-4d0e-9a83-201804c9adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:15.441143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f5e23-6fc1-4d0e-9a83-201804c9adc1 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.441678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f5e23-6fc1-4d0e-9a83-201804c9adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.442153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f5e23-6fc1-4d0e-9a83-201804c9adc1 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.443382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f5e23-6fc1-4d0e-9a83-201804c9adc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:15.445648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f5e23-6fc1-4d0e-9a83-201804c9adc1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:15.470574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932ea00-f036-4f4b-a610-e857314a038e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:15.483928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 360/713] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:18:15 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:18:15.512801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:15.513527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.513813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.514090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.514990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=862f9782f8574b8bb99d38c51333277f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:15.515306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:15.516958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e227b8-8a0c-438c-8ee8-ca2c91a32939 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:15.523973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 354/714] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:15 2022] GET /identity/v3/role_assignments?user.id=862f9782f8574b8bb99d38c51333277f&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:15.745906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ebea-ea92-4c6d-9c46-731eb9f4d22f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.746292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ebea-ea92-4c6d-9c46-731eb9f4d22f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.746466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ebea-ea92-4c6d-9c46-731eb9f4d22f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.748295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 361/715] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:15.755341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:15.755341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:15.755341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:15.773430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:15.774647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:15.813115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:15.855329 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 355/716] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:15.879074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18ebea-ea92-4c6d-9c46-731eb9f4d22f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:15.975494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:16.005960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18ebea-ea92-4c6d-9c46-731eb9f4d22f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:16.041273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:16.042416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:16.042699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:16.042976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:16.043629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:16.045190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:16.057277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:16.058339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:16.059200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 362/717] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:15 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:16.064719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:16.211430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c16069-abb4-4f51-93f2-8f5c3c44e338 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:16.271410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:16.272083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:16.272367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:16.272642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:16.273364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:16.273704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:16.275073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99c082-a80b-4b0f-9b14-6136ee27246e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:16.281479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 356/718] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:16.287177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:16.433345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53c3e8-6ce7-4ab7-bf15-0d62ed7b07f2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:16.479944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:16.480610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:16.480889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:16.481159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:16.482124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:16.482436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:16.483806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759b574-747b-4ccf-be98-cc3ee48a7fa1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:16.492615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 363/719] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:16.843798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352664-87f1-4ebe-a484-1574b5dc5f98 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:16.845356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352664-87f1-4ebe-a484-1574b5dc5f98 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:16.845882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352664-87f1-4ebe-a484-1574b5dc5f98 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:16.847693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 357/720] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:16.852810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:16.853344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:16.853857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:16.859240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:16.873117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:16.874711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:16.944098 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 364/721] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:16.970291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352664-87f1-4ebe-a484-1574b5dc5f98 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:17.036451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:17.130997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352664-87f1-4ebe-a484-1574b5dc5f98 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:17.180066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:17.180868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:17.181149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:17.181435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:17.193034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:17.200938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:17.201995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:17.202870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:17.206689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 358/722] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:17.208605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:17.344540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19138853-6c72-4a89-9e9e-350ab778eba9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:17.374821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:17.386094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:17.386709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:17.386995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:17.387278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:17.388066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:17.388381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:17.389855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0d9fe-47d6-4deb-ad7a-f4a2ae702993 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:17.398904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 365/723] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:17.404865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:17.517753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61bee4-1c65-4cea-9769-5b76b125e1c7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:17.518109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61bee4-1c65-4cea-9769-5b76b125e1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:17.518313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61bee4-1c65-4cea-9769-5b76b125e1c7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:17.519936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 366/724] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:17.527083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:17.529764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:17.530220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:17.558671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:17.562786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:17.576993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:17.587204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ebc3a-285e-42c3-a57f-b1658698f375 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:17.630699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:17.631905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:17.634023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:17.634479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:17.656818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:17.660089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:17.695519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:17.696520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:17.697517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:17.697517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:17.697517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:17.697517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:17.700082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 359/725] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:17.705589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:17.722612 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 367/726] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:18:17.767973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:17.817732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e286384-7b25-4389-905e-a00ebe0afa87 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:17.857015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:17.858397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:17.859287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:17.859709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:17.860578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:17.861025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:17.862850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:17.876367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 360/727] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:17.881386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:17.893782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:17.992832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3f68bb-d156-4d18-a75d-792a90e83457 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.036116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.037260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.037731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.038154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.047715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:18.048328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7406a-996f-401d-b2e2-a04202fb1464 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.049962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:18.067346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:18.068608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:18.069874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:18.072944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 361/728] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:18.078867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.123464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.147537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886f9bf-6740-455a-9b1d-3548b4a83cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.148209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886f9bf-6740-455a-9b1d-3548b4a83cc8 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.148531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886f9bf-6740-455a-9b1d-3548b4a83cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.148761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886f9bf-6740-455a-9b1d-3548b4a83cc8 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.149522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886f9bf-6740-455a-9b1d-3548b4a83cc8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:18.151025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886f9bf-6740-455a-9b1d-3548b4a83cc8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:18.173434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 368/729] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:17 2022] POST /identity/v3/services => generated 251 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:18:18.177106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.203013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e8c7-9b0a-4935-837b-894d715fca02 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.203305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e8c7-9b0a-4935-837b-894d715fca02 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.203586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e8c7-9b0a-4935-837b-894d715fca02 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.205373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 369/730] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:18.215067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0c339-3fd2-4e16-a2c3-b4961b383b48 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.215354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0c339-3fd2-4e16-a2c3-b4961b383b48 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.215702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0c339-3fd2-4e16-a2c3-b4961b383b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.235752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aa17d-e85f-4802-834d-0bd984c81fdb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.248676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-10c0c339-3fd2-4e16-a2c3-b4961b383b48 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:18.251896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-10c0c339-3fd2-4e16-a2c3-b4961b383b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:18.305576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.306263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.306547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.306819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.307742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:18.308055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:18.309678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e282b-eec2-4f59-90f5-6ba0ce2fdd0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:18.329706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 362/731] 10.222.0.24 () {56 vars in 1135 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:18.331327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080549f-0a61-4ab4-a7ca-2a3edce5bdfb None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.331618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080549f-0a61-4ab4-a7ca-2a3edce5bdfb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.331892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080549f-0a61-4ab4-a7ca-2a3edce5bdfb None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.337431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 363/732] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:18.338485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080549f-0a61-4ab4-a7ca-2a3edce5bdfb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.340169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 370/733] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 13:18:18.342304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.342594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.342878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.361766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:18.363055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:18.440798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 371/734] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:18.441695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.464542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080549f-0a61-4ab4-a7ca-2a3edce5bdfb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.488845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.506277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.513718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.514208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.514495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.514768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.580882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07aacf-c393-4b8c-bd1d-c575967a5c8b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.589107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:18.591096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:18.630095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.630940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.631228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.631506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.647362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:18.658856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:18.659829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:18.660699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 372/735] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:18.661456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.702183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 364/736] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 13:18:18.703385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.733216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.795991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558e838-dc47-48a3-a612-b17ecdf6fa68 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.798734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.835256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4602-4c76-420a-8ecd-42619434f650 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:18.839459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.840570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.841144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.841677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.855449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:18.865587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:18.866678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:18.867555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:18.875720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 373/737] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:18.875720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.884620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:18.886049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:18.886597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:18.887067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:18.888214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:18.888763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:18.890646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:18.900205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 365/738] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:18.901278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:18.987532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:18.995938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1bd18-6f5c-4751-a028-2c422a78eada None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.009665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ff70-e9f7-429a-9585-07da97b36b89 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.038102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.039321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.039614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.039895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.048132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.049111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.049626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.050191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.051139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:19.051739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.053292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.053796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.055262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.067939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 366/739] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:19.069126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:19.074791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:19.075783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:19.076707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:19.076707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:19.076707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:19.076707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:19.081717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 374/740] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:19.082300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.103863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.181025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350499c-d5d2-498b-9849-5a19d497052a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.196971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9c2a8-642d-4bf4-8459-2aaeff2e5baf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.217830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.218965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.219448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.219996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.231833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.233838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.235250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.236147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.236437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.236718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.240620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.253291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.254848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:19.261399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:19.262319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:19.263889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:19.263889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:19.263889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:19.263889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:19.270576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 367/741] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:19.271538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:19.284137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:19.285152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:19.286146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:19.286146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:19.286146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:19.286146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:19.292539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 375/742] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:19.292539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.367585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.403286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660155db-42b3-4da3-be5e-695d9b339dc8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.415012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b79d-14f6-45b3-88c3-f1328049071c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.438888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.439666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.440012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.440393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.441188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:19.441599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.443057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.459478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.460392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 368/743] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:19.460710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.461171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.461665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.461974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.462925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:19.463444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.465476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.481139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 376/744] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:19.482532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.531646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.579071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b5939-2519-4bca-9617-62e87d725ca9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.586648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeafac4-69be-48d2-a6ff-58121ef8d8ed None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.615624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.616023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.616393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.616679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.616949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.618109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:19.618455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.619874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecc6b1-5163-4d97-b806-e8434c0d9f24 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.625214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.626058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.626345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.626625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.631940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 369/745] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:19.633426 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.633931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17bb1-731f-4062-b0d9-a1b14bd1d1ee None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.634347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17bb1-731f-4062-b0d9-a1b14bd1d1ee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.634745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa17bb1-731f-4062-b0d9-a1b14bd1d1ee None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.634932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.636775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 370/746] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:19.637571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa17bb1-731f-4062-b0d9-a1b14bd1d1ee None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:19.656441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:19.657396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:19.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:19.658818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 377/747] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:19.659551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.750994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa17bb1-731f-4062-b0d9-a1b14bd1d1ee None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.763009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9bde7-573e-4143-8d4b-521ca7291348 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.767185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.785986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.786881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.787158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.787424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.796023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.797356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.797494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.798180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.798462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.798756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.806197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.807640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f4622f9c-6199-4268-b2ae-00df56904d49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:19.811925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:19.812889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:19.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:19.817837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 371/748] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:19.818866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.819150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.819421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:19.823803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:19.824753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:19.825841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 378/749] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:19.827379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.841778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:19.842981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:19.878519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:19.914179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 372/750] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 13:18:19.915025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:19.933679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783cf34-ab09-423f-a7e9-bddcf9d5d7d9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:19.969348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:19.970057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:19.970345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:19.970619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:19.971410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:19.971718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:19.973158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:19.986164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 379/751] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:19.986970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.037945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5e450-3f2d-47f9-afc9-8656335808e2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.046269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.073199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.074032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.074159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.074424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.075486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:20.075915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.077220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.099944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 373/752] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:20.099944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.100218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a63c60-9116-4d5e-9e2f-12b8b2b1671a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.109855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.137726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.138814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.139244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.139667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.140781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:20.141235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.143055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.166485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 380/753] 10.222.0.24 () {56 vars in 1149 bytes} [Thu May 19 13:18:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:20.166485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.210605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a069782-e64a-4220-94b7-a877e39f36ce None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.245645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.246310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.246590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.247003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.248037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.249727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.265589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 374/754] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:20 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:20.266372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.283328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186278-1e1e-4a61-b8b7-04b497f2ec1e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.297563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.336796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.337575 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.337861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.338141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.339033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=78b2f9c966e24c46a23631b80a578862) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:20.339417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.341000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.354311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 381/755] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/role_assignments?user.id=78b2f9c966e24c46a23631b80a578862&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:20.354953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.405023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.419928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c828ad6-7e48-4dab-be4f-f7d58d5b276d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.466952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.468256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.468749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.469228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.484852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.486749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:20.508279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:20.509363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:20.510331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:20.510331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:20.510331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:20.510331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:20.517640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 375/756] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:20.519630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.585121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e0db-da6c-4c6f-afe0-0461907c4345 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.628701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed466999-e44c-4695-8276-c0c74e0ffb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.630334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed466999-e44c-4695-8276-c0c74e0ffb95 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.630857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed466999-e44c-4695-8276-c0c74e0ffb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.631335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed466999-e44c-4695-8276-c0c74e0ffb95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.634702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.653873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3b536-cc44-463e-8167-804af4ca63c0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.689051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.689679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.689965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.690240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.690969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:20.691274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.692644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1125d2-cb76-4c73-ab65-ee2193731b73 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.707530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 376/757] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:20.713376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087539f-fa50-45e9-a92b-83f7d79bf64f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.713715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087539f-fa50-45e9-a92b-83f7d79bf64f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.713996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087539f-fa50-45e9-a92b-83f7d79bf64f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.715576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 377/758] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:20.716306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087539f-fa50-45e9-a92b-83f7d79bf64f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.721244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed466999-e44c-4695-8276-c0c74e0ffb95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.723103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed466999-e44c-4695-8276-c0c74e0ffb95 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.756451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.833390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087539f-fa50-45e9-a92b-83f7d79bf64f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:20.852899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 382/759] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:18:20.854175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.854458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.854735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.871491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:20.872950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:20.877449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:20.878444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:20.878868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:20.879279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:20.882875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:20.890738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:20.892869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:20.911667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:20.912836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:20.914264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:20.917343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 378/760] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:20.918265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:20.961110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 383/761] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:20.962021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.027423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.053314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7581-4a54-4b24-bcf6-09782901c22d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.080761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8b0c-7702-413c-b396-c5e8ff058f8d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.096763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.097828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.098108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.098381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.113492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.115080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.117238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.118076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.118418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.118748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.119653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:21.120095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.121715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.134699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 384/762] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:21.134876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:21.136157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:21.137879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:21.137879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:21.137879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:21.137879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:21.138170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.141153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 379/763] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:21.141996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.208152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.259610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac74e1c-fa13-4279-895f-23ec40f48d7a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.276764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.283411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd882a-d063-49d8-bc99-3b2f634acadf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.296492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.297305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.297615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.297891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.298544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.299920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:21.312586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:21.314750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:21.316802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:21.318746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 385/764] 10.222.0.24 () {56 vars in 1129 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:21.318746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.325175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.325175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.325175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.325175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/862f9782f8574b8bb99d38c51333277f/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.325885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=862f9782f8574b8bb99d38c51333277f, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.325885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.341676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 380/765] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:21 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/862f9782f8574b8bb99d38c51333277f/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:18:21.350595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.429358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746aea5-01a2-454a-8b30-19c8bde8a67d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.468988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.470316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.470601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.470879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.471624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:21.471929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.473334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.493547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 386/766] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:21.495195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.546419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.588732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc9eb1-2053-4f69-b218-2c7fad5789aa None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.610504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.619550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7e1f-4ae9-43fa-b4b3-27304bca0412 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.629757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.630437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.630826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.631109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.631828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:21.632240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.633659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.638936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 381/767] 10.222.0.24 () {56 vars in 1124 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:21.647212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.665119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.665981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.666258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.666539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.675498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.677041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:21.692586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:21.693839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:21.694792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:21.697172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 387/768] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:21.705802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.769029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ab8a4-b5a8-4292-afb7-e3d8bbe8aa7e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.804400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.804984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.805263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.805569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.806282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:21.806598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.807939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.818997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 382/769] 10.222.0.24 () {56 vars in 1124 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:21.820227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.855693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.882438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:21.886010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0886612-245f-4dc4-8807-08728264a677 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.923109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1670fb4-23bf-48ad-9e38-63c7955c0312 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:21.957263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.958374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.958852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.959381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.960568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] RBAC: Adding query filter params (interface=public, service_id=e850bb45c7c34902b715252dd4c192d7, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:21.961092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.962849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:21.975531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 383/770] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/endpoints?service_id=e850bb45c7c34902b715252dd4c192d7&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:21.976983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:21.981234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:21.981600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:21.981893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:21.982174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:21.982984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:21.983302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:21.984750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:22.024180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 388/771] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:22.026304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:22.127047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:22.130523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e1a69-ea43-4e9b-8128-bdebf323e954 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:22.166542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7de70f-aac1-4b4c-a052-86c6af40d83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:22.167702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7de70f-aac1-4b4c-a052-86c6af40d83a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:22.168203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7de70f-aac1-4b4c-a052-86c6af40d83a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:22.168971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7de70f-aac1-4b4c-a052-86c6af40d83a None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:22.169903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7de70f-aac1-4b4c-a052-86c6af40d83a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:22.171456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7de70f-aac1-4b4c-a052-86c6af40d83a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:22.181806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 384/772] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:21 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:22.233730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4e3d3-ebf2-4599-8ab3-3d763517127e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:22.295389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:22.296083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:22.296375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:22.296653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:22.297534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=e4d792d3845649cf9f28c529b67a2188) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:22.297855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:22.299333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397264e-7fc1-4541-99a2-43daeb839ac9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:22.310932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 389/773] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:22 2022] GET /identity/v3/role_assignments?user.id=e4d792d3845649cf9f28c529b67a2188&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:22.830187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9b65-1661-43c3-8917-8fae661aa5df None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:22.830603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9b65-1661-43c3-8917-8fae661aa5df None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:22.830793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9b65-1661-43c3-8917-8fae661aa5df None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:22.832531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 385/774] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:22.838566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221f55d-f3b8-44b1-aa31-390167b3bf68 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:22.838877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221f55d-f3b8-44b1-aa31-390167b3bf68 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:22.839122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221f55d-f3b8-44b1-aa31-390167b3bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:22.862443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7221f55d-f3b8-44b1-aa31-390167b3bf68 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:22.863638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7221f55d-f3b8-44b1-aa31-390167b3bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:22.888733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:22.956075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 390/775] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:22.989703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c9b65-1661-43c3-8917-8fae661aa5df None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.044940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320512d-29ad-41be-83ea-6e983692b2b0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.045303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320512d-29ad-41be-83ea-6e983692b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.045539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320512d-29ad-41be-83ea-6e983692b2b0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.047059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 391/776] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:23.052202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.052499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.052780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.070102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:23.071316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:23.155916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 392/777] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:23.157445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c9b65-1661-43c3-8917-8fae661aa5df None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.180732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.233831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.234638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.234924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.235200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] PATH_INFO: `/v3/services/network` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.235867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:23.237254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:23.254841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:23.255922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:23.256789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 386/778] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:22 2022] GET /identity/v3/services/network => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:23.266419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.308499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:23.325675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516c117-7d62-48d8-a7e5-c33f1dd5485c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.345872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:23.360966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.362133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.362673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.363153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.374608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:23.382630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:23.383658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:23.384529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:23.387326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 393/779] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:23.393293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.423533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3296a7f-f121-4d07-a7c2-961636feecbe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.483668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.484390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.484675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.484949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.485901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] RBAC: Adding query filter params (name=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:23.486009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:23.487362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:23.497559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 387/780] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:23.499158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38f47d-883d-48dd-b357-733cfaea4c83 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.502743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.535047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.536095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.536576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.537051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.538075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:23.538601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:23.540491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:23.549937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 394/781] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:23.555789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.565654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:23.658938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d5309-16fb-4bb0-92ea-a592894f149e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.693382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.694546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.695041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.695528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.706499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:23.708220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:23.717788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:23.726043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:23.726976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:23.727905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:23.727905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:23.727905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:23.727905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:23.730148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 395/782] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:23.735958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.741626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eabf06-c0a2-4064-9d10-7aa40d7ea971 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.802039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:23.846380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa074a72-7690-45ae-99e8-0a0ae2ebba11 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:23.883532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.884560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.885044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.885555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.886110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:23.886749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:23.886868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:23.887054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:23.887332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:23.887789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:23.888081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] RBAC: Adding query filter params (type=network) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:23.888389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:23.889772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:23.889996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:23.898733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 388/783] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:23.904268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 396/784] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:23.908180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:23.911223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.029796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145de984-4198-4496-b7ff-1a53ccc6c364 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.053616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fa214-771e-454b-acfc-16db0e24298d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.055429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:24.078788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.080277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.081040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.081910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.089436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.090112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.090390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.090849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.091715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] RBAC: Adding query filter params (interface=public, service_id=bdb1aef6f7fa45cda1b394d8caa8691e, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:24.092028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.092784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.093380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557437fe-fb7e-40c8-a731-fcc10150fd49 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.094974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.103276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 389/785] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/endpoints?service_id=bdb1aef6f7fa45cda1b394d8caa8691e&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:24.104637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427b89a-d595-4743-82dc-2fd9399b0123 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.104921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427b89a-d595-4743-82dc-2fd9399b0123 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.105190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427b89a-d595-4743-82dc-2fd9399b0123 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.110670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 390/786] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:24.111413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427b89a-d595-4743-82dc-2fd9399b0123 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a924f01e-f4bd-48b0-9e5e-760d2ff958a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:24.112689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:24.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:24.114638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:24.118307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 397/787] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:24.123381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.123381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.123381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.140800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:24.148364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:24.150447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:24.229474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427b89a-d595-4743-82dc-2fd9399b0123 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.245143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 398/788] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:24.246397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.285861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.286378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.286659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.286930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.287631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.289326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.301845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 391/789] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:24.302813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.371462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1930-9f06-4419-9a98-1fe814ab4dc5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.383761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:24.414925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.416597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.417088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.417997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.419165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:24.419783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.421490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.445748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 399/790] 10.222.0.24 () {56 vars in 1135 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:24.452172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.527604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:24.558191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371f3aa-3a8e-494a-ad36-04207938640f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.598587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.599186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.599514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.599786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.611283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:24.618680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:24.619672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:24.620533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 392/791] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:24.625979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.627650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f1132-676a-43b0-b329-2a0eb9609fa4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.674888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.676338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.676840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.677467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.694252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.695716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:24.714849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:24.715931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:24.716851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:24.716851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:24.716851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:24.716851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:24.716851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 400/792] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:24.722929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.741030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e536c-2b4a-4b86-97d9-d170ea8b14ae None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.748884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:24.778674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.779397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.779682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.779957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.781030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:24.781140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.782746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.793209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 393/793] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:24.800822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:24.812069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:24.916761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69756919-9b9f-4801-911e-7507cc35b84a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.951630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa5fb5-8711-4727-a618-f0d54ffbfaf9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:24.954217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:24.955251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:24.955725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:24.956210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:24.969524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:24.971323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:24.989667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:24.990613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:24.991526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:24.991526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:24.991526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:24.991526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:24.994018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 394/794] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:25.000251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.030973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.031661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.031941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.032215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.033013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:25.033265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.034742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c6d07-67aa-4645-8585-4bbcf3cb50e8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.061554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 401/795] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:25.063189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e4ce0-a4b7-475f-a6d7-c2f73056970f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.063421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e4ce0-a4b7-475f-a6d7-c2f73056970f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.063703 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e4ce0-a4b7-475f-a6d7-c2f73056970f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.065941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 402/796] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:25.066839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e4ce0-a4b7-475f-a6d7-c2f73056970f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.074288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.118741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ea464-d454-48b7-baf7-249fb9284691 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.149876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.156353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.157269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.157747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.158211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.159193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:25.159644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.161156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8263d-5c6f-41c5-a8cc-7e634c6976b6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.172361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e4ce0-a4b7-475f-a6d7-c2f73056970f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.176177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 395/797] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:25.177725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38cad4-4c08-4ce3-bb60-65a8de4cb8a9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.178273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38cad4-4c08-4ce3-bb60-65a8de4cb8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.178745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38cad4-4c08-4ce3-bb60-65a8de4cb8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.197326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1c38cad4-4c08-4ce3-bb60-65a8de4cb8a9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:25.199103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1c38cad4-4c08-4ce3-bb60-65a8de4cb8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:25.208736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.209578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.209864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.210140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.217362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.218839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:25.233694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:25.234818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:25.235938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:25.239497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 403/798] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:25.240383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.284152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 396/799] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:25.285851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0a36-885a-404b-9b38-43f0e3306a66 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.286206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0a36-885a-404b-9b38-43f0e3306a66 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.286555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0a36-885a-404b-9b38-43f0e3306a66 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.288039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 397/800] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:25.292838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e0a36-885a-404b-9b38-43f0e3306a66 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.350044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.364125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a44c60-99e3-4d17-90c2-85acce1b5977 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.400746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.401588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.401867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.402140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.406461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e0a36-885a-404b-9b38-43f0e3306a66 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.409643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.411109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.412959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-acc77514-8b52-4bce-89e0-20a62fff0f5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:25.427124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:25.428123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:25.429110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:25.432925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 404/801] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:25.434533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.434829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.435107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.445130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.446222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.446509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.446788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.447584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:25.447913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.449359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.451323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:25.452506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:25.466019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 398/802] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:25.466019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.526458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 405/803] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:25.527162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.581310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52954c-a97c-4dc4-9ace-10976fd2d8b5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.591752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.620972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.621876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.622159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.622433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.627407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee8de5-2c7b-4998-ad33-948c92b5280d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.633731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:25.642411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:25.643567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:25.644638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:25.646821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 399/804] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:25.647733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.659541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.669403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.670193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.670476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.670754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.671599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:25.671908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.673345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.691266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 406/805] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:25.692122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.762838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392674-d841-4c75-a4df-c38106856ec1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.797964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.798883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.799170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.799446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/78b2f9c966e24c46a23631b80a578862/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.801503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=78b2f9c966e24c46a23631b80a578862, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.801871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69aa5a-984a-416e-ab53-658b39037577 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:25.802986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.829445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 400/806] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:25 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/78b2f9c966e24c46a23631b80a578862/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:18:25.829445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.855828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:25.858503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:25.858870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:25.859217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:25.859973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:25.861920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:25.864914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:25.880406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:25.881456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 13:18:25.882343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:25.885919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 407/807] 10.222.0.24 () {56 vars in 1129 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:25.887164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:25.947994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:25.977503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41441d35-3d59-492c-a024-6ce25840b6c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.008106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a6944-7fb3-4b95-8018-8f7498a98bc5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.025874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.026895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.027288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.027629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.029170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:26.029595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.031045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.043176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 401/808] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:26.044305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.050516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.051418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.051898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.052348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.142204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.144402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.153433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:26.167336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a1195-69da-4fd4-b004-8b91cdc068a6 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.213685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:26.215168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.215947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.216237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.216507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.217231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:26.217562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.219066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.226261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 402/809] 10.222.0.24 () {56 vars in 1124 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:26.227501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.283134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 408/810] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:18:26.284113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.351446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af24fe3-f96f-4ad7-86c3-9aa1d35a5b02 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.385528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.386318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.386597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.386867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.392790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224150-48c9-4ec8-a2bb-64aff81546d2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.398059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.399511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:26.417583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:26.418588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:26.419543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:26.419543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:26.419543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:26.419543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:26.423504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 403/811] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:26.424350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.429785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.430451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.430733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.431009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.431724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:26.432029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.433606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:26.434182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.445348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 409/812] 10.222.0.24 () {56 vars in 1124 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:26.446326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.465994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:26.527466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5627d-2b0b-4fda-b131-0652ae6e75d0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.554120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884a644-b874-4f39-bd12-cf53cd0c652d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.573150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.573972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.574248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.574516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.588387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.589981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.597604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.598302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.599080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.599528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.600546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:26.600860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.602403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:26.612691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:26.614724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:26.616768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:26.616768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:26.616768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:26.616768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:26.617275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 410/813] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:26.617275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.621840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 404/814] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:26.623135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.700436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:26.739855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:26.746924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc729369-397e-4e08-964f-d095cc6ed4b9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.771855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d616fa-bb7e-49a3-bd89-c2da13ae3386 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:26.789054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.790138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.790426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.790703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.791434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:26.791748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.793681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582005c0-85a5-44ca-8ab8-3d3d40c007e9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.807444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:26.808263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.808699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.809068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.810158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:26.811741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:26.848818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 411/815] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:26.849974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:26.964678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 405/816] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:26.966647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6945a-ccb9-4d65-90c6-4df358f76e45 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:26.967035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6945a-ccb9-4d65-90c6-4df358f76e45 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:26.967404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6945a-ccb9-4d65-90c6-4df358f76e45 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:26.969000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 406/817] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:26.973248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6945a-ccb9-4d65-90c6-4df358f76e45 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.000444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e134df-71f4-4977-ad41-7c48675be35f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.036618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.048909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.049825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.050156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.050439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.060040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.061808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ef3ea855-7baa-4b74-9229-37b5666ff65c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:27.078689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:27.079739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:27.080687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:27.085240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 412/818] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:27.087125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.087303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.087448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.094685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6945a-ccb9-4d65-90c6-4df358f76e45 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.108279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:27.109553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:27.116587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.141358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.142957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.143247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.143651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.151963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.153436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:27.169392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:27.170857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 407/819] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:27.171825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.205282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 413/820] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:27.206279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.313999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33cbe06-0910-40eb-ba8b-669eb214f1f4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.340870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeda50d-4013-4be0-8ab8-2e5a1771274c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.356969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.358229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.359027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.359399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.360577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:27.361238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.362259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.364071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.385826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.386590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.386951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.387321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.387450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 408/821] 10.222.0.24 () {56 vars in 1149 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:27.388271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.389941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:27.390260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.393591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.409860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 414/822] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:27.410681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.431570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.505024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf914790-a08c-4473-a12e-bd8753bc25dc None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.538391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b44a-370f-48b4-a54f-b58625edae8d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.549899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.550950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.551483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.551970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] PATH_INFO: `/v3/services/network` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.552828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.554524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:27.568012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:27.569133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 13:18:27.570081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:27.572846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 409/823] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:27.573871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.580672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.581608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.581982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.582362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.593792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.595507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:27.613610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:27.614791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:27.615957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:27.615957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:27.615957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:27.615957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:27.621630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 415/824] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:27.622675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.651179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.688849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.692956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361e8c8-927d-4ef1-960b-4b1fce597bda None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.726631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240769cf-57c2-44af-a69f-27b428ac3310 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.731163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.732020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.732453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.732945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.734004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=862f9782f8574b8bb99d38c51333277f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:27.734463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.736149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.749527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 410/825] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/role_assignments?user.id=862f9782f8574b8bb99d38c51333277f&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:27.750913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.764105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.764874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.765494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.765631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.766344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] RBAC: Adding query filter params (name=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:27.766787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.768159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.773342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 416/826] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:27.779519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.869287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d5d61-fc44-4919-aa59-018a1c77c223 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.907693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.908684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.909172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.909670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.910624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:27.911113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.912745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.926812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 411/827] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:27.929803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.938021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d4704-7bfd-4e8c-b4e8-269ebcc3e77b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:27.938608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:27.966770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:27.976720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:27.977391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:27.977721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:27.977999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:27.978716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] RBAC: Adding query filter params (type=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:27.979012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:27.981551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:27.990266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 417/828] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:27.996334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:28.068886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d9765-6227-49b3-9ac6-99f2b3ed7471 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.113968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.114770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.115048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.115838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.120041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893b1ab-fcc9-4f41-9db4-c2f31922cc05 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.124554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:28.125955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:28.142914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:28.143881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:28.144884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:28.150018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 412/829] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:28.157043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:28.173671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ce805-a7c5-467a-b94f-987d6b144c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.174207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ce805-a7c5-467a-b94f-987d6b144c9d None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.174491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ce805-a7c5-467a-b94f-987d6b144c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.174770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ce805-a7c5-467a-b94f-987d6b144c9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.175521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ce805-a7c5-467a-b94f-987d6b144c9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:28.176907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ce805-a7c5-467a-b94f-987d6b144c9d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:28.203966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 418/830] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:28.205219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610930fc-53b1-4d95-9465-76a5eab7a0ba None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.205623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610930fc-53b1-4d95-9465-76a5eab7a0ba None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.205905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610930fc-53b1-4d95-9465-76a5eab7a0ba None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.207368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 419/831] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:28.213063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.213353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.213671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.231958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:28.233239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:28.247940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:28.288450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:28.320589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76096-f99a-4181-99dc-c032188b2680 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.349996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 420/832] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:28.378489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.379075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.379355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.379625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.380424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:28.380731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:28.382205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:28.389069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:28.408409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 413/833] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:28.414764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:28.562514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da51529-7d35-4eea-b33c-9b0413a289e1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.563853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310093d8-f52b-4175-87e0-4bdb7760c6c9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.605568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.605829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.606588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.606868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.607135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/e4d792d3845649cf9f28c529b67a2188/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.607783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.608272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.608750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.618770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:28.625627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=e4d792d3845649cf9f28c529b67a2188, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:28.626409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:28.627076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:28.629526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:28.636832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:28.638225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:28.639117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:28.642053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 421/834] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:28.644982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 414/835] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:28 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/e4d792d3845649cf9f28c529b67a2188/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:18:28.650897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:28.773616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2421bc-88e9-469e-9da0-5d9ba87f4c77 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.813515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.814580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.815083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.815593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.816561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:28.817076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:28.818710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:28.828804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 422/836] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:28.834966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:28.944380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a8beb-a9fb-49e3-9725-74e383ea5897 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:28.978898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:28.979688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:28.979964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:28.980234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:28.990975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:28.992509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:29.012021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:29.013902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:29.014832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:29.014832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:29.014832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:29.014832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:29.014832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 415/837] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:29.019314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.082176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:29.128300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905041f-a921-4135-930a-059f3912fc90 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:29.164385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:29.165434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.165926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.166457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.167415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:29.167938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:29.170145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.183042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 423/838] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:29.192910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.250360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:29.315712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9b4ea-4e87-4696-8b26-0d8d106362ba None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:29.354530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:29.355335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.355613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.355880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.364126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:29.365593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-07656801-9d83-4f10-b79e-c6c177bebe46 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:29.380739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:29.381926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:29.383254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 416/839] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:29.389800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.408282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab4d7-f281-4709-bb93-516d59330ddf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.408615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab4d7-f281-4709-bb93-516d59330ddf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.408840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab4d7-f281-4709-bb93-516d59330ddf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.410516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 417/840] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:29.415854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.416138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.416411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.433307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:29.434556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:29.497986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8978a9-69c7-40d6-9a66-10f9f2cd38b4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:29.503366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 418/841] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:29.518519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:29.525917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.532573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:29.533252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.533561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.533838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.534680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:29.534988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:29.536441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.556667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 424/842] 10.222.0.24 () {56 vars in 1135 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:29.562379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.570616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:29.638061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3337b7a-3527-47da-b996-ec0167fe4be2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:29.676162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:29.676960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.677237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.677539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.678202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:29.679565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:29.691326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:29.692327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:29.693235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 419/843] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:29.697079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c5fb1-1609-4542-85a0-2a1c9541fbb5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:29.698263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.737533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:29.738182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.738460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.738730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.787614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:29.812751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:29.814372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.828727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:29.832995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95b66-cded-4d3d-be19-724694ddea3a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:29.880335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:29.881686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:29.882424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:29.883510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:29.884681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:29.885199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:29.887045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:29.894452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 420/844] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:29.903596 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:29.935295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 425/845] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:18:29.936992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.020188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:30.044907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb99a1-557b-4aac-a9db-f15a07589031 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.051301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda51684-bb28-440b-8611-7a9032d586bb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.081681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.082483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.082744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.083093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.093842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.095349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.099396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.100298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.100783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.101267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.102259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:30.103253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.105456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:30.117452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:30.118454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:30.119384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:30.119384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:30.119384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:30.119384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:30.120088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 426/846] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:30.120541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 421/847] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:30.120905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.238327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984f4f8-8dd6-4c17-aa14-aa46df9d3368 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.271744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:30.278213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.278897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.279184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.279468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.280210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:30.280519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.281969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.295753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 427/848] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:30.305158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.339421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:30.414962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30235abc-ee0d-4b9a-aedd-6ec3605e63eb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.449459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.450329 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.450612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.450888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.458051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.459500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5b019a84-b733-4bcf-990f-1274fb53b726 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:30.474191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:30.475135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:30.477153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 422/849] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:30.481175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.495987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c78fe-17d2-417f-81be-94172624eb5c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.496288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c78fe-17d2-417f-81be-94172624eb5c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.496537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c78fe-17d2-417f-81be-94172624eb5c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.498212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 423/850] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:30.503244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.503623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.503895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.522705 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:30.523891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:30.592972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed845d-e10a-4708-9559-9f54278ccd1e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.611847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 424/851] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:30.635877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.637539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.638327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.638613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.638990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.639793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:30.640105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.641581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.655192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:30.656314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 428/852] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:30.662311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.717032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:30.741273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610988f-497e-4bc9-a282-f0661e7acf36 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.771119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8336f-d2d4-4c1b-8963-0276c55b2d49 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.777104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.778081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.778402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.778721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.779483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.780844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:30.794641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:30.796816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:30.798911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:30.805850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 425/853] 10.222.0.24 () {56 vars in 1132 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:30.805850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:30.806714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.807374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.807656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.807929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.808763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=78b2f9c966e24c46a23631b80a578862) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:30.809074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.810601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.822629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 429/854] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/role_assignments?user.id=78b2f9c966e24c46a23631b80a578862&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:30.905913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:30.923658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2feed-7e8d-431b-83c7-1ba5fef8c60d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:30.960612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:30.961294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:30.961610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:30.961885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:30.962743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:30.963052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:30.964439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d35d1-9c39-4f72-ad6e-881912af33a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:30.970437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 426/855] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:30.978553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:31.039382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df712e9-c8b3-4a52-96a7-23dc8e052b7f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.039756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df712e9-c8b3-4a52-96a7-23dc8e052b7f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.040032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df712e9-c8b3-4a52-96a7-23dc8e052b7f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.041593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 427/856] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:31.047682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.047967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.048242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.070139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:31.071657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:31.090213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6350531-e1aa-44e2-b5be-ca896329f2c3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:31.106081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:31.138513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:31.139730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.140451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.140915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.141845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:31.142338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:31.144316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:31.151280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 430/857] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:31.159734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 428/858] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:31.183142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:31.300131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455b777-9667-4b96-b2c3-bba4982d48c6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:31.338875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:31.339684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.339964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.340237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.352221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:31.360294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:31.361330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:31.362469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 431/859] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:18:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:31.369813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:31.454013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:31.477767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899b070-e8dc-4489-936a-3d8913ce8440 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:31.516468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:31.517187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.517490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.517780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.518532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:31.518832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:31.520187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:31.529246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 429/860] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:18:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:31.534769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:31.578717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:31.640030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e0184-6ca3-4bf3-96a3-d6023e7434f0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:31.674422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:31.675272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.675552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.675876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.686836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:31.688365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:31.707112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:31.708084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:31.709463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:31.709463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:31.709463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:31.709463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:31.709463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 432/861] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:31.715721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:31.820965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4c5f-0fc5-4786-973e-a75a666e3fcf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:31.854798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:31.855460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:31.855738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:31.856238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:31.856871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:31.857778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:31.858551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:31.868806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:31.871153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 430/862] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:31.877123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:31.980795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:31.987996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d558ce3-51eb-41b4-a634-92e321ca1fd2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:32.023735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:32.024547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.024825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.025094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.033290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:32.034769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:32.050227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:32.051262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:32.052458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 433/863] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:18:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:32.059514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:32.167538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f83e1-705a-4c40-a963-225591558c8b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:32.202331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:32.203023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.203299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.203570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.204437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:32.204746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:32.206548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835222fd-9234-4c1c-af82-a725104bca1c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:32.227721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 431/864] 10.222.0.24 () {56 vars in 1149 bytes} [Thu May 19 13:18:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:32.234814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:32.341198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e2c8b-267a-4d26-8cc2-378731db0d79 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:32.375158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:32.375801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.376081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.376350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.397490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd11871-a70d-41be-b390-6bbb8c5875c1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.398013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd11871-a70d-41be-b390-6bbb8c5875c1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.398013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd11871-a70d-41be-b390-6bbb8c5875c1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.399577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:32.400028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 432/865] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:32.405322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.405921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.406145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.429251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:32.430937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:32.451686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:32.453164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:32.493461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:32.513635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 433/866] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:32.536963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:32.565063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 434/867] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:18:32.570257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:32.630325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:32.659923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a458c-f1f9-4cc9-adc1-cba36c572d5b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:32.684046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10afd8-b137-419f-a320-a2d4023314c5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:32.695867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:32.696562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.696826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.697106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.697882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:32.699252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:32.720797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 434/868] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:32 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:32.731074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:32.732293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.732786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.733268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.746515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:32.748609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:32.769498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:32.770512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:32.771453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:32.771453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:32.771453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:32.771453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:32.778208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 435/869] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:32.780420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:32.833632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:32.904285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea5ae8-a99f-45b4-b347-8862b848450a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:32.944076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:32.945121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:32.945705 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:32.946178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:32.947121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:32.947632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:32.949218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:32.964044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 435/870] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:32.968285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.080765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:33.085211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc287dba-f4d0-4a29-b398-a8e9aaf4ace2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.120347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.121573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.122822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.123329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.134149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.137027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c4b891c0-9701-44ef-9d94-3efd258edc14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:33.151417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:33.152376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:33.153461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 436/871] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:33.160519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.197459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5bdbf-c4f0-4e3b-ab41-00bb80612322 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.197840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5bdbf-c4f0-4e3b-ab41-00bb80612322 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.198022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5bdbf-c4f0-4e3b-ab41-00bb80612322 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.200181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 437/872] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:33.205544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.205707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.205981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.224825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:33.226247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:33.268695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1a7be-5a32-4d57-85e5-62bf8e364dda None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.277107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:33.305799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.306851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.307364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.307849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.308839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:33.309655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.311441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.317555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 438/873] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:33.324365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 436/874] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:33.330468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.340247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.449678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1a6f3-4d2e-4af5-bcbb-579351154510 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.450538 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fd5dc-3a86-446e-9ba8-8b08177a1924 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.453770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:33.488865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.490056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.490521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.491495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.492181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=e4d792d3845649cf9f28c529b67a2188) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:33.492774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.493782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.494679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.495315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.495598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.495870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.496566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.498850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.509850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 439/875] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:33 2022] GET /identity/v3/role_assignments?user.id=e4d792d3845649cf9f28c529b67a2188&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-74168334-ce0a-4baa-a48f-ad54e9dea28f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:33.512189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:33.513585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:33.514484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 437/876] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:33.521989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.550910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0d20-9c79-4adb-94dc-f2a94e73dff1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.551213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0d20-9c79-4adb-94dc-f2a94e73dff1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.551459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a0d20-9c79-4adb-94dc-f2a94e73dff1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.553001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 438/877] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:33.558711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.558976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.559411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.580607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:33.583731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:33.614799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:33.648534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef4509-aa46-43fd-81ba-0ba7adc9e7d7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.685570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 439/878] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:33.706690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.707786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.708299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.708779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.709750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:33.710260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.711797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.716547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.723681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 440/879] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:33.730591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:33.763570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:33.860184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552548-90d6-41f5-9cf2-5576c0317ee7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.874663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651e36f-d9d5-4688-a318-8f4d7e844393 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:33.903181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea831551-f5b3-4f05-94ad-7bc5584253cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.904722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea831551-f5b3-4f05-94ad-7bc5584253cc None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.905238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea831551-f5b3-4f05-94ad-7bc5584253cc None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.905810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea831551-f5b3-4f05-94ad-7bc5584253cc None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.907068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea831551-f5b3-4f05-94ad-7bc5584253cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.909564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea831551-f5b3-4f05-94ad-7bc5584253cc None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.923345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:33.924077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:33.924362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:33.924687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:33.925430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:33.925764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:33.927435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bc2b8-7d62-429c-b3ce-eb0de1b31b36 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:33.933648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 441/880] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:33.936116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 440/881] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:33 2022] POST /identity/v3/services => generated 269 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:18:35.213931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fb2bf-14b2-4faf-92c0-17950b01b797 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.214496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fb2bf-14b2-4faf-92c0-17950b01b797 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.214496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fb2bf-14b2-4faf-92c0-17950b01b797 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.216572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 442/882] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:35.222936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.223506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.223506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.241821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:35.243018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:35.258871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:35.324128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 441/883] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:35.346621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fb2bf-14b2-4faf-92c0-17950b01b797 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:35.431887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:35.452301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fb2bf-14b2-4faf-92c0-17950b01b797 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:35.486408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:35.487221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.487502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.487777 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.488435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:35.489847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:35.502080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:35.503147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:35.504025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 443/884] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:35.510632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:35.612526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff300e1-69f5-47a8-a01b-4a715acc6503 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:35.647278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:35.648267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.648723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.649191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.650114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:35.650595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:35.652123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:35.657707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 442/885] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:35.663591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:35.770258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5231-12a8-4bac-9151-5b0ec3fb5d9c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:35.808137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:35.809216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.809733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.810261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.811221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:35.811727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:35.813339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7472989-d06b-4123-8330-c128ccac0bd5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:35.820004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 444/886] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:35.826748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:35.842040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:35.940447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1214cb-91d5-4290-9478-23e028813a3d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:35.977396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:35.978091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.978376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.978653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.979458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] RBAC: Adding query filter params (interface=public, service_id=8555455045c3434a8cef48994b21a29d, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:35.979776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:35.981647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:35.987831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 443/887] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:35 2022] GET /identity/v3/endpoints?service_id=8555455045c3434a8cef48994b21a29d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:35.991618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807c858-332a-490e-b8de-2c915d988bf0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:35.991916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807c858-332a-490e-b8de-2c915d988bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:35.992200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807c858-332a-490e-b8de-2c915d988bf0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:35.993831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 445/888] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:35.997448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.000347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.000866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.001307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.021890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:36.023462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:36.026100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:36.085695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:36.116796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 446/889] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:36.125673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dd2ed-30b1-4e79-a893-08ac16a55470 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:36.136519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.166515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:36.167576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.168105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.168395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.169190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:36.170865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:36.179549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 444/890] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:35 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:18:36.273666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918344f-98ae-4b60-a119-08c4b565826d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:36.329276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbd189-b159-455a-879d-16c9c84b3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:36.330613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd189-b159-455a-879d-16c9c84b3c76 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.331113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd189-b159-455a-879d-16c9c84b3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.331600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd189-b159-455a-879d-16c9c84b3c76 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.344334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:36.352268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbd189-b159-455a-879d-16c9c84b3c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f7fbd189-b159-455a-879d-16c9c84b3c76 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:36.365864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:36.366934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:36.367811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:36.370829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 447/891] 10.222.0.24 () {56 vars in 1120 bytes} [Thu May 19 13:18:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:36.379621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.415657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bea43-e48a-48dd-9fc6-617c5068e514 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.415965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bea43-e48a-48dd-9fc6-617c5068e514 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.416216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bea43-e48a-48dd-9fc6-617c5068e514 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.421396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 448/892] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:36.424254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdcbfd-90c4-4533-82f8-41be116ad2be None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.424543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdcbfd-90c4-4533-82f8-41be116ad2be None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.424818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdcbfd-90c4-4533-82f8-41be116ad2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.453927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-38bdcbfd-90c4-4533-82f8-41be116ad2be None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:36.455113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-38bdcbfd-90c4-4533-82f8-41be116ad2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:36.528420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274a0ac-9c98-4a4f-a315-25c6385a58bf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:36.553220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 449/893] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:36.553220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35515929-e03a-4f05-bda5-f625b13b0cb9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.553220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35515929-e03a-4f05-bda5-f625b13b0cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.553220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35515929-e03a-4f05-bda5-f625b13b0cb9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.553220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 450/894] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:36.558689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.558689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.559075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.576009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:36.576940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.577366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.577974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.578910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:36.579563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:36.581181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:36.584790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:36.586020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:36.587254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:36.592540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 445/895] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:18:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:36.593478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.678955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 451/896] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:36.678955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.715243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9caa08-6118-4a9f-8d79-d2f3ee1d6df5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:36.758576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:36.759231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.759513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.759788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.760681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:36.761950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:36.763749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:36.778123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 446/897] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:36 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:36.779332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.822947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a27305-1ec6-4fcc-96cc-fa7007a48e84 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:36.855816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:36.868392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:36.869203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.869551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.869878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.886690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:36.888367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:36.908324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de686-1d2c-448e-b696-d8a3aa303430 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:36.909008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:36.910104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:36.911511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:36.911511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:36.911511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:36.911511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:36.911511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 452/898] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:36.916475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:36.960102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:36.961486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:36.962035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:36.962531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:36.963435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:36.965031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:36.978949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:36.979980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:36.980847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:36.982527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 447/899] 10.222.0.24 () {56 vars in 1132 bytes} [Thu May 19 13:18:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:36.990373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.049364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0936b69-5015-477a-9157-f7ff9c4be12a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.060935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.093888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.095242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.095529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.095803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.096801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:37.097106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.099460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.107423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626581c2-a84e-42cb-9c4d-81bdd8fdd75c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.114758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 453/900] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:37.120106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.135731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.147662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.148413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.148762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.149103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.150049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:37.150434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.151802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.164122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 448/901] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:37.165438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.238517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ffbd5-853e-4351-b9bb-758e78cd01c8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.285010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.286496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.286784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f7302-c8c8-4eb1-a34d-a0ed9c4ba4af None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.287091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.287504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.298314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.300662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:37.321024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:37.322262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:37.323460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:37.325809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.330334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.330334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.330334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.330334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:37.330334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.330334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.331104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 454/902] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:37.332441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.340872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 449/903] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:37.342315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.368114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.407405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.457264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90bb6-c408-4952-aa91-cc2ebd9fb426 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.474130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b696f69-7fbb-4cbd-bd95-045b97a9d0a4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.494555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.495236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.495539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.495957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.496918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:37.497233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.498924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.517571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.522135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 455/904] 10.222.0.24 () {56 vars in 1149 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:37.523161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.523672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.524286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.524770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.525998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] RBAC: Adding query filter params (interface=public, service_id=51f57d930f6a4c7fb1cfe175ff06a9f1, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:37.526516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.529018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.535838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 450/905] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/endpoints?service_id=51f57d930f6a4c7fb1cfe175ff06a9f1&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:37.545514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.616164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.634244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a30867-aa6d-4f66-bb27-be5241929f64 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.670049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa96779-0e6a-48b4-be51-8e10db8f0c46 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:37.671699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.672640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.672987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.673393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.691359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.716680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:37.718225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:37.718695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:37.719599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:37.720544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.722825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.732260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 451/906] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:18:37.761440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:37.763031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:37.860180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:37.880189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 456/907] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:18:37.885720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:37.995260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48559c5-6fa0-4450-a855-914f9ba13a7a None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:38.038408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:38.039522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.039806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.040080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.051700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:38.053146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:38.070968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:38.071975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:38.072946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:38.072946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:38.072946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:38.072946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:38.072946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 452/908] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:38.078491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:38.179475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60df8d-7cc6-4a4a-917f-365cca08613e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:38.214616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:38.215318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.215697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.216160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.216892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:38.217205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:38.218619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:38.230493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 457/909] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:38.235904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:38.284757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:38.354442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c33c58-0d39-4643-b83d-b21204915c10 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:38.392796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:38.394052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.394580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.395070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.404553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:38.406518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-33140367-e68e-4e9d-b5d0-b9ffaeab1243 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:38.424150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:38.425639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:38.426571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:38.429283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 453/910] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:38.437506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:38.481867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:38.541009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd586329-6f3a-4590-a66c-5862d49a7de3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:38.576019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:38.576687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.576968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.577241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.578068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:38.578378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:38.579820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:38.584692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79598-2a37-4835-ac5d-13a0bc23d33c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.584977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79598-2a37-4835-ac5d-13a0bc23d33c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.585248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79598-2a37-4835-ac5d-13a0bc23d33c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.586886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 454/911] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:38.592153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.592677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.593137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 458/912] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:38.593378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.599555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:38.614644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:38.616016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:38.701752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 455/913] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:38.709902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ada171-0330-4856-ac3a-352466a2cd19 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:38.712316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:38.721526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:38.747060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:38.747731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.748039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.748320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.749149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] RBAC: Adding query filter params (role.id=0bb38eca871c4a7f9e6e24200e62626a, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=e4d792d3845649cf9f28c529b67a2188) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:38.749485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:38.750932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:38.754577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:38.758806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 459/914] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:38 2022] GET /identity/v3/role_assignments?user.id=e4d792d3845649cf9f28c529b67a2188&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0bb38eca871c4a7f9e6e24200e62626a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:38.854408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c0313-d093-403f-8ba0-2642e96000ea None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:38.895374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:38.896198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:38.896485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:38.896758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:38.897462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:38.898879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:38.912825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:38.914281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:38.915159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 456/915] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:38.923121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.040036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a03c5-556a-4530-ae83-34331f12566e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:39.073862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:39.074527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.074844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.075110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.075928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:39.076234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:39.077624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5e075-70e8-4416-b507-2fa59326ba4d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:39.082830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 460/916] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:39.088110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.181910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:39.199066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5335f-effc-4ed0-a566-4168b66b8d84 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:39.206945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08732e-cb0c-4d61-a048-ba7eb41f67c2 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.207236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08732e-cb0c-4d61-a048-ba7eb41f67c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.207522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08732e-cb0c-4d61-a048-ba7eb41f67c2 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.208911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 461/917] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:39.215157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.215582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.215910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.236635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:39.238277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:39.239687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:39.242733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.243863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.243863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.244182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:39.244500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:39.245920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:39.252214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 457/918] 10.222.0.24 () {56 vars in 1114 bytes} [Thu May 19 13:18:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:39.264703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.270739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:39.321604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 462/919] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:39.342457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.371192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbb16a-16d8-4cb4-b94a-4a43ec20ed12 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:39.407454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551134ad-bc46-4473-896d-437255014a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:39.407927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551134ad-bc46-4473-896d-437255014a78 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.408214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551134ad-bc46-4473-896d-437255014a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.408493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551134ad-bc46-4473-896d-437255014a78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.409260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551134ad-bc46-4473-896d-437255014a78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:39.410692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551134ad-bc46-4473-896d-437255014a78 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:39.436121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 458/920] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:39.455070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198651e-7cbf-4896-8e8d-443e477e0204 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:39.502250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:39.503452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.503935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.504429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.505300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:39.507010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:39.520486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:39.521632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:39.522514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:39.529799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 463/921] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:39.532447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551134ad-bc46-4473-896d-437255014a78 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.543221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:39.654126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551134ad-bc46-4473-896d-437255014a78 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:39.689664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:39.690382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.690670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.690946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.691701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:39.691971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:39.693348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:39.699027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 459/922] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:39.705184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.732334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:39.810097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dd409-40c4-49f2-8018-0634f10f61ef None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:39.844372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:39.845044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:39.845328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:39.845635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:39.846343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:39.846649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:39.848087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:39.857518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 464/923] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:39.862516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:39.942067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:39.976415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdefc92-d5b1-453e-a8ec-b439ca832e56 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:40.011907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:40.012654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.012953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.013259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.014096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] RBAC: Adding query filter params (interface=public, service_id=c55b411d70554d57866a174d4f090c3a, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:40.014405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:40.015774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e1297-a917-4cd0-8c4f-005453677678 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:40.025556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 460/924] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:39 2022] GET /identity/v3/endpoints?service_id=c55b411d70554d57866a174d4f090c3a&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:40.027271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:40.116635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:40.119072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392455-52ae-4bcc-92be-7217b4eaef64 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.119355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392455-52ae-4bcc-92be-7217b4eaef64 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.119635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3392455-52ae-4bcc-92be-7217b4eaef64 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.121274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 461/925] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:40.127262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.127677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.128056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.136642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e0963-a280-46ab-9f75-c9b4a4d30df7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:40.148657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:40.150284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:40.174009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:40.175005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.175294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.175576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.176266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:40.177735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:40.189439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 465/926] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:40 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:18:40.247203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 462/927] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:40.270968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:40.383879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efc255-cb99-43a0-8305-86d3e03cec1b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:40.420204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:40.421380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.421885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.422358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.423208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:40.424813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:40.440285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:40.441278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 13:18:40.442291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:40.449607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 466/928] 10.222.0.24 () {56 vars in 1132 bytes} [Thu May 19 13:18:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:40.454532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:40.518126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:40.555810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb927e0-ded8-4ab9-b696-0f23cf5ae724 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:40.589526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:40.590197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.590537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.590816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.591737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:40.592049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:40.593440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:40.598833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 463/929] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:40.604353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:40.622282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:40.712321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb9d2c-58fe-407b-8c31-e7605c5e8fb0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:40.747066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:40.747802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.748087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.748364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.749082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:40.749393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:40.750806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47b1fd-6904-4fa5-ba81-f4860c2255b6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:40.761421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 467/930] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:40.764148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:40.866228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c353-b6b8-46aa-a8ef-cf3252fe3ad9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:40.897506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:40.906140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaf057-3f26-488b-85c1-2a6b1ed343ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:40.906798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf057-3f26-488b-85c1-2a6b1ed343ce None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:40.907075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf057-3f26-488b-85c1-2a6b1ed343ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:40.907345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaf057-3f26-488b-85c1-2a6b1ed343ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:40.908096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaf057-3f26-488b-85c1-2a6b1ed343ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:40.909481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbaf057-3f26-488b-85c1-2a6b1ed343ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:40.931252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 464/931] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:18:41.182935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933cbd-091c-4211-b9f6-9c237677506a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.183139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933cbd-091c-4211-b9f6-9c237677506a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.183423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933cbd-091c-4211-b9f6-9c237677506a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.185055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 468/932] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:41.190648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.190940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.191219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.211002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:41.212179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:41.286663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 465/933] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:41.309284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933cbd-091c-4211-b9f6-9c237677506a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:41.391442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:41.417007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933cbd-091c-4211-b9f6-9c237677506a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:41.451732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:41.452473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.452756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.453031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.463907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:41.472137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:41.473119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:41.474019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 469/934] 10.222.0.24 () {56 vars in 1120 bytes} [Thu May 19 13:18:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:41.479401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:41.584952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ae506-7584-42d5-955b-84c95244b0bb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:41.596772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:41.626203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:41.627264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.627746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.628203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.629152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:41.629666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:41.631286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a2d10-c4b1-40a6-9f1e-34e47d1e59b1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:41.645865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 466/935] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:18:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:41.647836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:41.751554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0f4b4-3f3d-4502-9456-a2a4e90b6e1e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:41.785768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:41.786627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.786911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.787226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.794600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:41.798920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:41.800425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:41.815450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9d995-4069-4f96-a801-b8955e5deac9 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.816200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9d995-4069-4f96-a801-b8955e5deac9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.816695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9d995-4069-4f96-a801-b8955e5deac9 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:41.818534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:41.819469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:41.820414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:41.820414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:41.820414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:41.820414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:41.820414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 470/937] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:41.821284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 467/937] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:41.827577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:41.829458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.829987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.830503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.849453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:41.850995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:41.933436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 468/938] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:41.937945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6695e24-c661-4f3d-bff3-4766fbcffca7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:41.951885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:41.984603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:41.985299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:41.985582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:41.985861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:41.986586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:41.986894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:41.988291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.002472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 471/939] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:42.010705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.066767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:42.092668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e08cbb-3727-4e8d-8f3c-4d65112ad5fa None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.094404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:42.131386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.132044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.132321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.132590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.133284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.134939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.136352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING keystone.server.flask.common [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 13:18:42.140265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038adbf-2ab4-4214-a17b-8119e8366c78 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.157381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 13:18:42.157381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 13:18:42.157381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [parameters: {'id': 'c0f7b2372c984c5db33c3af9db52b94f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 13:18:42.157381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0f7b2372c984c5db33c3af9db52b94f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0f7b2372c984c5db33c3af9db52b94f. May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:18:42.171996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 13:18:42.172901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() May 19 13:18:42.173922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 13:18:42.174865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 13:18:42.175799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:18:42.176653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'id': 'c0f7b2372c984c5db33c3af9db52b94f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.181793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:42.182646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c0f7b2372c984c5db33c3af9db52b94f. May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.183624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 469/940] 10.222.0.24 () {60 vars in 1140 bytes} [Thu May 19 13:18:41 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:42.187067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.202991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.204422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.205002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.205701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.215966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.218927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:42.239851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:42.240938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:42.241960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:42.249484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 472/941] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:42.251382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.311472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da91dff-ce38-432c-86cf-05021e901f77 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.347309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.347552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.347821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.347996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.358940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.361049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:42.377279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:42.378356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.379283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 470/942] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:42.385459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.396008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eaaf3-ecd8-4504-9889-52ab82a0aa10 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.408656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:42.415271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:42.439816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.440588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.440942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.441287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.442239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:42.442620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.444145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.464948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 473/943] 10.222.0.24 () {56 vars in 1149 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:42.470276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.524139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe6e0-daef-4b9b-a43e-19fac0898e94 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.566978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.567643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.567925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.568199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.568964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:42.569274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.570742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d7fc6-6454-4f93-927d-eb9117a7a636 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.592513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b2a81-2f80-415b-8bc0-b2f33c083bd3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.594108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 471/944] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:42.595557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b877b20-c981-4ecd-85cd-ce25cbb08ef3 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.595851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b877b20-c981-4ecd-85cd-ce25cbb08ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.596128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b877b20-c981-4ecd-85cd-ce25cbb08ef3 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.601680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 472/945] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:42.603714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.603714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.603918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.625851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:42.627446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:42.646733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.647548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.647985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.648515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.663352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.665454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:42.689569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:42.690557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:42.691499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:42.691499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:42.691499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:42.691499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:42.691737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 474/946] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:42.696477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.711300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 473/947] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:42.726586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:42.736529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.749445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:42.837512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b413d16-4dcd-46ab-9281-7004a6d6b226 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.861780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190a203-6cba-44f4-8aae-05d129eaa441 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:42.890013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.890013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.890013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.890013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.891657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:42.891657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.893284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.899280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:42.900094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:42.900375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:42.900647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:42.901309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:42.904231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:42.907961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 475/948] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:42.918770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:42.919237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:42.920276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:42.921149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:42.924215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 474/949] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:42.929958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.010180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:43.023108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:43.047319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8a0bd-1395-4f6d-ac19-21a7c6292e4d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.048866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d72f7-8c57-44b3-a650-3548f760b28d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.086897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.088031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.088613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.089175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.090272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:43.090760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.092232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.092617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.093163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.093570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.093924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.099474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 475/950] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:43.106094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.106297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.108460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:43.125528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:43.126525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:43.127649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:43.135362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 476/951] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:43.136775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.234687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894e75c-aeca-44d3-b8ae-946f6c690298 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.255873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4fc71-7d5e-48c8-afb3-4a7f36be5f7c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.271476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.272425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.272890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.273341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.274275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:43.274757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.276294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.278851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:43.283353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 476/952] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:43.290710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.305989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.307696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.308049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.308393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.309338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:43.309848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.311498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059615de-3737-46a6-a304-14e3d85fd108 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.321669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:43.332148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 477/953] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:43.333590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79746-72af-47b7-9d53-259859e29519 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.333953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79746-72af-47b7-9d53-259859e29519 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.334303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c79746-72af-47b7-9d53-259859e29519 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.337833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 478/954] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:43.337833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79746-72af-47b7-9d53-259859e29519 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.401171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fec03-24b3-4611-a25e-b9cdebc744f2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.436164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720e092-e00a-4baf-8142-7acef66ccd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.436785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720e092-e00a-4baf-8142-7acef66ccd5b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.437067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720e092-e00a-4baf-8142-7acef66ccd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.437343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720e092-e00a-4baf-8142-7acef66ccd5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.438113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720e092-e00a-4baf-8142-7acef66ccd5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.439478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720e092-e00a-4baf-8142-7acef66ccd5b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.447190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c79746-72af-47b7-9d53-259859e29519 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.460491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 477/955] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:43.461797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.462085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.462359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.481968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:43.483191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:43.504101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.505521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.506006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.506471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/e4d792d3845649cf9f28c529b67a2188/roles/0bb38eca871c4a7f9e6e24200e62626a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.509319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=e4d792d3845649cf9f28c529b67a2188, role_id=0bb38eca871c4a7f9e6e24200e62626a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.511205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.538118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 479/956] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:18:43 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/e4d792d3845649cf9f28c529b67a2188/roles/0bb38eca871c4a7f9e6e24200e62626a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:18:43.559294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:43.572871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 478/957] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:43.598372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.667664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:43.723899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5b301-75aa-4715-a557-b1ad26f69419 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.761187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.762399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.762834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.763256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.764079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.765652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:43.778988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:43.780398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:43.781610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:43.784229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 480/958] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:43.790357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:43.900662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70734b05-698e-4801-b4c5-6697f480a4e0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:43.937078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:43.938024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:43.938453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:43.938898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:43.939757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] RBAC: Adding query filter params (name=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:43.940225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:43.941761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1fff6-1fa3-45af-b253-c3ac2965ffda None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:43.947548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 479/959] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:43.953100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:44.057278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b418e6e-0879-4f04-9b46-67b5b78e7c71 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:44.078059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:44.093881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:44.094547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:44.094826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:44.095098 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:44.095806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:44.096110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:44.097496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730356d1-a5be-4ede-8bb1-96e0c2b2bba8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:44.102561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 481/960] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:44.967426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3ff23-9a8b-460c-99a6-b1cfcb4ccd96 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:44.968402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3ff23-9a8b-460c-99a6-b1cfcb4ccd96 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:44.968890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3ff23-9a8b-460c-99a6-b1cfcb4ccd96 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:44.970830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 480/961] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:44.976498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:44.977290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:44.977803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:44.995176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:44.997104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:45.076189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 482/962] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:45.101297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3ff23-9a8b-460c-99a6-b1cfcb4ccd96 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:45.159028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:45.214010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3ff23-9a8b-460c-99a6-b1cfcb4ccd96 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:45.265787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:45.267172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.267586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.268055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.269230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:45.271584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:45.292868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.sql.core [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 13:18:45.292868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 13:18:45.292868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [parameters: {'user_id': '32705de4ec8f41b5ac3fd0ce176d09cd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 13:18:45.292868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80249) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() May 19 13:18:45.307773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:18:45.308735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application yield resource May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application next(self.gen) May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.rollback() May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.force_reraise() May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise self.value May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application session.commit() May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 13:18:45.309854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._prepare_impl() May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.session.flush() May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._flush(objects) May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application compat.raise_( May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application flush_context.execute() May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rec.execute(self) May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 13:18:45.311317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 13:18:45.313710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result = self._query(query) May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application conn.query(q) May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application result.read() May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 13:18:45.317885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [parameters: {'user_id': '32705de4ec8f41b5ac3fd0ce176d09cd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:45.318762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 13:18:45.319649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:45.320614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 481/963] 10.222.0.24 () {60 vars in 1131 bytes} [Thu May 19 13:18:45 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:45.321099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:45.425294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a6522-eb16-4824-84af-adb4a20a447b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:45.440653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:45.461549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:45.462490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.462860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.463208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.470954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:45.472416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b66b88b2-88c9-45de-b47c-1f18d3bbaf8e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:45.487265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:45.488268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:45.489240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 483/964] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:45.494752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:45.597143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04545b6a-6110-428a-a977-0ef9641df07c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:45.617982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:45.634647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:45.635349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.635662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.635922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.636810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:45.637177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:45.639056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e1ee-0819-42dc-99a0-2163c043cb98 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:45.681457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 482/965] 10.222.0.24 () {56 vars in 1101 bytes} [Thu May 19 13:18:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:45.813196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6c6d-68ae-42c8-9976-468b82114203 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.813578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6c6d-68ae-42c8-9976-468b82114203 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.813790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b6c6d-68ae-42c8-9976-468b82114203 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.815490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 484/966] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:45.821456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.821802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.822037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.840130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:45.841956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:45.873362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f930864-c0d7-4af3-b025-7d3ef53f369f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.873578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f930864-c0d7-4af3-b025-7d3ef53f369f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.873845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f930864-c0d7-4af3-b025-7d3ef53f369f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.875336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 485/967] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:45.880997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:45.881279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:45.881576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:45.903041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:45.903955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:45.927433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 483/968] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:45.953136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:45.987684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 486/969] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:46.011814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.078311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.081400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0dd74-906f-44ba-988b-f268cde1103e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.118165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0c511-4c98-4fb4-bb43-81b7da3f268b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.119963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.120470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.121783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.122368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.122824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.133601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:46.142192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:46.143335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:46.144370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:46.146794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 484/970] 10.222.0.24 () {56 vars in 1120 bytes} [Thu May 19 13:18:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:46.152617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.160561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.161360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.161672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.161928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.162610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.164325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:46.177009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:46.178100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:46.179086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 487/971] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:46.184939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.272914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05c111-4b69-4804-ad3b-9ebbdebfae11 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.306444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb99fc-c9f9-4983-ac37-37ea9f8466c5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.311068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.311740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.312020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.312300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.313046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:46.313352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.315183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.324663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 485/972] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:46.331662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.348920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.349638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.349900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.350173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.350882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:46.351279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.352663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.357647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 488/973] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:46.359738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.363130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.397537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.441632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77fa5-7581-4234-acb4-13718532676f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.472171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13302aee-572e-4735-8778-dfaba0654bcd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.474656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.475424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.475695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.475962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.486810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.488274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-70e209e2-9bad-4566-b1e0-34af2d398ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:46.505197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:46.506373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:46.513258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:46.513258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:46.513258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:46.513258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:46.513644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.513644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.513644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.513937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 486/974] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:46.514024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.514024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:46.514024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.514256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025664e-f137-441d-956a-ec4e11d438ed None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.514256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025664e-f137-441d-956a-ec4e11d438ed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.514256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025664e-f137-441d-956a-ec4e11d438ed None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.514748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6e84f-eb59-470e-9e7b-b4d6db57f5b5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.514868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 487/975] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:46.514868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025664e-f137-441d-956a-ec4e11d438ed None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.515073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 489/976] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:46.515706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.515990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.516265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.532051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:46.533233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:46.593690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.599962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 490/977] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:46.600901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.627794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.633662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025664e-f137-441d-956a-ec4e11d438ed None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.666460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.667053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.667333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.667603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.668403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:46.668784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.670369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.681899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 488/978] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:46.682654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.708933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846e4db-cce0-4c9f-b1aa-a78ab2a50805 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.743330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.743989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.744274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.744557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.745364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] RBAC: Adding query filter params (interface=admin, service_id=c55b411d70554d57866a174d4f090c3a, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:46.745725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.747315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.754863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 491/979] 10.222.0.24 () {56 vars in 1249 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/endpoints?service_id=c55b411d70554d57866a174d4f090c3a&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:46.755572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.782182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1efe69-0609-4b3d-9a8f-e841c2aaaacd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.815894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.816555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.816836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.817111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.817868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.819237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.821502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.833517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 489/980] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:46 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:46.834424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.846213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:46.861151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb374-508f-40a6-baaa-b8a62003b079 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:46.907027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:46.907861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:46.908146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:46.908427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:46.917286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:46.918947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:46.934645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:46.935728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:46.936753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 492/981] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:46.941952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:46.963334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8de30-d68f-4657-bbc2-1c8ed50c7206 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:47.002058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:47.002803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:47.003093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:47.003368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:47.004125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:47.005568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:47.012662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 490/982] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:18:47.063003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f1691-e19d-4c83-84b7-2c090e76e7f9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:47.102155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:47.102838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:47.103120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:47.103395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:47.104262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:47.104568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:47.106071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:47.127489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 493/983] 10.222.0.24 () {56 vars in 1149 bytes} [Thu May 19 13:18:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:47.132953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:47.198568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:47.248483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ad197-4c01-40da-9bbe-708dc02a4745 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:47.288232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:47.288892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:47.289525 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:47.289805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:47.361100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:47.362601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:47.426484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:47.472721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 491/984] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:18:47.478442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:47.482697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:47.592398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ed64b-e179-4c60-b88b-381be138d395 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:47.634479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:47.635285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:47.635972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:47.636253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:47.647584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:47.649210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:47.668453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:47.669696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:47.670657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:47.670657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:47.670657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:47.670657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:47.673507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 494/985] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:47.675812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:47.778128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f5b9-02f0-403b-b0b1-3acde253b51b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:47.813067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:47.813752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:47.814031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:47.814305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:47.815027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:47.815332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:47.816681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:47.829597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 492/986] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:47.834916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:47.900230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:47.948329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2726f4-96af-4d27-bcfd-8dc62d468b73 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:47.984079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:47.984862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:47.985199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:47.985510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:47.992380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:47.993872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:48.009241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:48.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:48.011323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:48.013526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 495/987] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:48.017250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:48.038498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:48.122815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381849-10e3-4f9a-bd2e-0242b56924bf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:48.159834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:48.161252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:48.161743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:48.162197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:48.163228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:48.163738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:48.165670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ff088-55e7-40e4-bd79-0db172c853a5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:48.183105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 493/988] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:48.185580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:48.301598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa360296-e682-49c3-a87a-1a3ce508868b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:48.320325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:48.340407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:48.341070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:48.341357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:48.341664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:48.342501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] RBAC: Adding query filter params (role.id=0bb38eca871c4a7f9e6e24200e62626a, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=e4d792d3845649cf9f28c529b67a2188) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:48.342816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:48.344265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd55c32-b77b-4b82-b627-2f700e6ef57c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:48.357718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 496/989] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:48 2022] GET /identity/v3/role_assignments?user.id=e4d792d3845649cf9f28c529b67a2188&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0bb38eca871c4a7f9e6e24200e62626a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:49.121452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64029-9165-4082-9b51-7b6011b90868 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.122235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64029-9165-4082-9b51-7b6011b90868 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.122658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64029-9165-4082-9b51-7b6011b90868 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.124019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 494/990] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:49.129684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09603bc-1263-4095-902f-b30d388c61b9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.129951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09603bc-1263-4095-902f-b30d388c61b9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.130254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09603bc-1263-4095-902f-b30d388c61b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.149253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b09603bc-1263-4095-902f-b30d388c61b9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:49.150471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b09603bc-1263-4095-902f-b30d388c61b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:49.224720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 497/991] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:49.247645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64029-9165-4082-9b51-7b6011b90868 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.264598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318a172-f6ab-488a-98af-994cff193ebe None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.264870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318a172-f6ab-488a-98af-994cff193ebe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.265154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318a172-f6ab-488a-98af-994cff193ebe None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.266682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 498/992] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:49.271954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.272250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.272529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.290228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:49.291436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:49.337491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:49.363615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64029-9165-4082-9b51-7b6011b90868 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.364660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 499/993] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:49.387420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.399319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.400128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.400408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.400682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.401346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.402891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:49.417766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:49.418957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:49.419926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 495/994] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:49.425211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.434059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:49.497447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326f6a3-de76-400e-a722-aa828dc0cbe3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.534589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.535397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.535679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.535954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.536574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afeef2-efd9-47da-86a3-2aa07a8eff24 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.536685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.538012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:49.550618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:49.551671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:49.552571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 500/995] 10.222.0.24 () {56 vars in 1126 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:49.557832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.577887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.578561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.578904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.579184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.579927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:49.580235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.581735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.592058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 496/996] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:49.592820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.602956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:49.664837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3008980-877e-4970-a144-551a73c036c2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.667501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:49.703705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.704380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.704664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.704942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.705695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:49.706009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.706108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d9c3f-d32e-4a07-be0c-cc152ec5d994 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.707453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.718423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 501/997] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:49.719741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.748840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.749539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.749821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.750093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.750795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:49.751096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.752450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.757249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 497/998] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:49.763276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.825733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c127522-1ef5-4565-8fc5-44a11bf3eb33 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.833820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:49.861503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.862175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.862463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.862901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.863616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:49.863924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.865373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.866141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c0935-b2df-44a2-9613-deb444d0800e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:49.870628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 502/999] 10.222.0.24 () {56 vars in 1122 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:49.876367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.905978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:49.906644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:49.906928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:49.907210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:49.908012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] RBAC: Adding query filter params (interface=public, service_id=db7f2745b8b44261adcbdf4229b80265, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:49.908327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:49.908456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:49.909719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:49.915862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 498/1000] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/endpoints?service_id=db7f2745b8b44261adcbdf4229b80265&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:49.921136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:49.985174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61683c-9e7e-44ba-be7b-45906c5add96 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:50.026302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae758138-b237-40e0-9aee-8a12327ef7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:50.026972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae758138-b237-40e0-9aee-8a12327ef7fa None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:50.027353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae758138-b237-40e0-9aee-8a12327ef7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:50.027599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae758138-b237-40e0-9aee-8a12327ef7fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:50.028614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae758138-b237-40e0-9aee-8a12327ef7fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:50.030066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae758138-b237-40e0-9aee-8a12327ef7fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:50.033312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0c428-9598-411d-9d0d-68367d8ca770 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:50.057456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 503/1001] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:18:50.085082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:50.093766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b926c3-ebcc-4337-aab4-8b7304acd770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:50.094450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b926c3-ebcc-4337-aab4-8b7304acd770 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:50.094733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b926c3-ebcc-4337-aab4-8b7304acd770 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:50.095009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b926c3-ebcc-4337-aab4-8b7304acd770 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:50.095692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b926c3-ebcc-4337-aab4-8b7304acd770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:50.097074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b926c3-ebcc-4337-aab4-8b7304acd770 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:50.104484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 499/1002] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:49 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:18:50.584661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233ff3-a2be-431f-8c7d-99fc901756eb None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:50.585174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233ff3-a2be-431f-8c7d-99fc901756eb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:50.585174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233ff3-a2be-431f-8c7d-99fc901756eb None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:50.586511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 504/1003] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:50.591987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:50.592430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:50.592507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:50.610164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:50.611353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:50.681261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 500/1004] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:50.702858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a233ff3-a2be-431f-8c7d-99fc901756eb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:50.799120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:50.810104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a233ff3-a2be-431f-8c7d-99fc901756eb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:50.845810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:50.846924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:50.847388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:50.847851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] PATH_INFO: `/v3/services/image` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:50.848677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:50.850269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:50.862256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:50.863313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:50.864199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:50.866389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 505/1005] 10.222.0.24 () {56 vars in 1105 bytes} [Thu May 19 13:18:50 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:50.872523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:50.977344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc332df-f6fc-4956-b112-d239e554d51b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:51.002709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:51.014563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:51.015231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:51.015513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:51.015787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:51.016492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:51.016798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:51.018189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0eb00-c24e-4bdc-9404-3a557317b734 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:51.023118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 501/1006] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:51.028741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:51.131343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e07152-d11b-4258-92b0-e1f2fc172143 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:51.166929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:51.167589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:51.167869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:51.168143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:51.168901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] RBAC: Adding query filter params (type=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:51.169204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:51.170713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b0687e-04d1-44ec-af70-8c66fa6a0524 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:51.175551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 506/1007] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:52.343355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd986e03-c64b-46c4-b423-13a2a183c0af None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.343895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd986e03-c64b-46c4-b423-13a2a183c0af None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.343895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd986e03-c64b-46c4-b423-13a2a183c0af None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.345472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 502/1008] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:52.347489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326eb96-ad3f-4a4d-b7dd-611d4deb0e0d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.347896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326eb96-ad3f-4a4d-b7dd-611d4deb0e0d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.348056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326eb96-ad3f-4a4d-b7dd-611d4deb0e0d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.349691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 507/1009] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:52.350860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.351141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.351412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.354573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.354864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.355143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.371203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:52.372732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:52.377324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:52.378586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:52.395336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:52.441736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 503/1010] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:52.452718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 508/1011] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:52.464166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:52.476058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:52.568079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77781ca9-3c5e-4262-8343-9b0ec52ec84c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:52.583064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f716fc-7670-4715-95d1-bd0e0293a624 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:52.587392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:52.606652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:52.607451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.607729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.608000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.609166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:52.610658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:52.617073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 504/1012] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:52.617905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:52.620489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:52.621297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.621681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.621962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.622625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:52.624006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:52.635938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:52.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:52.637946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 509/1013] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:52.642916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:52.659147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:52.734738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe02f7-7233-40d3-ab41-86d31d4bc603 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:52.749698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75a4ff-82af-4a5d-8ae0-e189afd289d6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:52.770754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:52.771432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.771711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.771984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.773114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:52.774746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:52.786960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:52.787629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.788024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.788309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.789457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:52.789890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:52.791500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:52.798748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 510/1014] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:52.802792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 505/1015] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:52 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:18:52.810668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:52.899920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:52.925093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7273-1673-4151-9358-0c8ad38d7c0e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:52.962791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:52.963466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:52.963750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:52.964024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:52.964745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:52.965105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:52.966609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80d63c-dff8-4633-bf59-1faa64fc725d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:52.972639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 511/1016] 10.222.0.24 () {56 vars in 1112 bytes} [Thu May 19 13:18:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:52.978219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:53.024144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:53.088961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60259d-d035-4ea2-a131-4b58d9275e04 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:53.131032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef49888-72c7-4e66-b5e8-48e47ddae926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:53.131700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef49888-72c7-4e66-b5e8-48e47ddae926 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:53.131990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef49888-72c7-4e66-b5e8-48e47ddae926 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:53.132264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef49888-72c7-4e66-b5e8-48e47ddae926 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:53.133021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef49888-72c7-4e66-b5e8-48e47ddae926 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:53.134427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef49888-72c7-4e66-b5e8-48e47ddae926 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:53.155415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 506/1017] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:53.363115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d71a6-3ec8-4c03-aa56-35ff38c8eef2 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:53.364169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d71a6-3ec8-4c03-aa56-35ff38c8eef2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:53.364457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d71a6-3ec8-4c03-aa56-35ff38c8eef2 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:53.366249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 512/1018] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:53.372417 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555639-491b-470a-b29f-e680bcbbed0c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:53.372912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555639-491b-470a-b29f-e680bcbbed0c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:53.373007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555639-491b-470a-b29f-e680bcbbed0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:53.391097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d6555639-491b-470a-b29f-e680bcbbed0c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:53.392280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d6555639-491b-470a-b29f-e680bcbbed0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:53.470541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 507/1019] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:53.492918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d71a6-3ec8-4c03-aa56-35ff38c8eef2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:53.595509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d71a6-3ec8-4c03-aa56-35ff38c8eef2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:53.631692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95feb9a-ac7f-4e22-8b0d-e913c3c584b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:53.632355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95feb9a-ac7f-4e22-8b0d-e913c3c584b6 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:53.632636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95feb9a-ac7f-4e22-8b0d-e913c3c584b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:53.632903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95feb9a-ac7f-4e22-8b0d-e913c3c584b6 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:53.633648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95feb9a-ac7f-4e22-8b0d-e913c3c584b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:53.635044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95feb9a-ac7f-4e22-8b0d-e913c3c584b6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:53.652096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 513/1020] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:18:53 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:55.085065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605d79-6e55-4471-975c-39b93e49e8b5 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.085628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605d79-6e55-4471-975c-39b93e49e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.085628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3605d79-6e55-4471-975c-39b93e49e8b5 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.087044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 508/1021] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:55.092676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.093105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.093278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.098457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:55.112747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:55.113965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:55.198536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 514/1022] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:55.222780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3605d79-6e55-4471-975c-39b93e49e8b5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:55.248434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:55.341083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3605d79-6e55-4471-975c-39b93e49e8b5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:55.377839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b41e7-4006-4649-abfa-cb143bc6b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:55.378631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b41e7-4006-4649-abfa-cb143bc6b537 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.378906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b41e7-4006-4649-abfa-cb143bc6b537 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.379176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b41e7-4006-4649-abfa-cb143bc6b537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.380392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b41e7-4006-4649-abfa-cb143bc6b537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:55.381868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b41e7-4006-4649-abfa-cb143bc6b537 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:55.383677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 509/1023] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:55.389568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:55.494372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd630075-ad33-4397-ad3c-723d90e38918 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:55.507604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89458d-6165-4503-811b-953396ca4d58 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.507912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89458d-6165-4503-811b-953396ca4d58 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.508163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89458d-6165-4503-811b-953396ca4d58 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.509588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 510/1024] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:55.514766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.515054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.515334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.526831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:55.533163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:55.534559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:55.541286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:55.542013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.542299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.542571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.543297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:55.544754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:55.568124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 515/1025] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:18:55 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:18:55.617754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 511/1026] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:55.642565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:55.765223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd2592-ad0f-4c33-94de-b9a35d67a906 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:55.801847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa02407-d10a-4985-b85b-b45d0abeb2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:55.802667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa02407-d10a-4985-b85b-b45d0abeb2ec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.802949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa02407-d10a-4985-b85b-b45d0abeb2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.803220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa02407-d10a-4985-b85b-b45d0abeb2ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.814880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa02407-d10a-4985-b85b-b45d0abeb2ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-efa02407-d10a-4985-b85b-b45d0abeb2ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:55.823007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:55.823973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:55.825319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 516/1027] 10.222.0.24 () {56 vars in 1120 bytes} [Thu May 19 13:18:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:55.831153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:55.852867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:55.894906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1331e-7c65-40cf-95b5-c36df3c36d7f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.895218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1331e-7c65-40cf-95b5-c36df3c36d7f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.895524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1331e-7c65-40cf-95b5-c36df3c36d7f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.896846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 517/1028] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:55.902529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.903724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.904321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.927762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:55.929569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:55.940089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9bcb0-0e1d-4654-918c-0449868fa438 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:55.977339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:55.978047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:55.978331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:55.978609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:55.979373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:55.979679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:55.981125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:55.981593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:55.990221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 512/1029] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:18:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:55.996126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.018942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 518/1030] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:18:56.041396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.087685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.108273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be60dca-7454-4c82-97b5-397b62dc2630 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.141995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.142786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.143067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.143339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.143435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81f001-58e7-45f0-9e9d-3c35b6b79ebd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.154663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.156032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:56.174104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:56.175103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:56.176029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:56.176029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:56.176029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:56.176029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:56.176029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 513/1031] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:56.180812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.185449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.186127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.186349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.186565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] PATH_INFO: `/v3/services/image` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.187232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.188623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:56.200655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:56.201755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:56.203480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 519/1032] 10.222.0.24 () {56 vars in 1105 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:18:56.207812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.258413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.301228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d38ba-1b8a-44e6-bc8f-c24b56acc46f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.320967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd2663-9271-435b-ba81-a2a6665a2a51 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.332070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.339221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.339893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.340278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.340708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.342161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:56.342483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.343858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.356582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.357237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.357660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 514/1033] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:56.361787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.362106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.362821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.362946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] RBAC: Adding query filter params (name=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:56.363212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.364634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.376102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 520/1034] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:56.377401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.467875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57bfc0-30ee-4d28-a7dd-a90bf9341a12 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.484554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b4ae0-e75f-4fbf-9acd-bf163ac17585 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.486242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.504259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.505368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.505688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.505964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.515376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.516841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.522216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.522891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.523178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.523453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.524202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:56.524510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.525993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:18:56.531633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:56.532614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:18:56.533649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:18:56.537802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 515/1035] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:56.538661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.541994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 521/1036] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:56.544285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.592644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.643462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e58bce-1c97-4e36-bb72-a39781c767f3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.653782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712e521-6d5a-4429-906f-090cb4c40fb7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.681566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.682247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.682526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.682797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.683646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:56.684223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.685431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.689064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.689750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.690036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.690316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.691104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] RBAC: Adding query filter params (interface=public, service_id=1868a7e61f884697b8e365effbe3304a, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:56.691419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.692788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.702964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 522/1037] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/endpoints?service_id=1868a7e61f884697b8e365effbe3304a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:56.703990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.707151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 516/1038] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:56.713821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:56.736384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.811286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f72c4-e45c-49ba-abb2-82c32ae5c7a1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.825695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa453d-89e5-4095-b195-de5e260192dd None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:56.837494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:56.850659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.851335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.851614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.852140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.852576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.853967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:56.873445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 523/1039] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:18:56.873667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:56.873667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:56.873667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:56.873667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:56.957938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:56.959405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:57.070778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 517/1040] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:18:57.076871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:57.182670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:57.192703 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b3a37-104e-47d7-bc49-47f3eb591eff None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:57.230193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:57.231095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.231429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.231672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.247334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:57.248812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:18:57.273362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:57.274346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:57.275294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:18:57.275294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:18:57.275294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:18:57.275294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:57.275294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 524/1041] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:57.280554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:57.292630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:57.386294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b4824-e227-43de-bddd-5bb100588da9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:57.422924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:57.423906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.424370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.424823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.425796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:57.426288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:57.427830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:57.440987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 518/1042] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:57.446367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:57.554607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e04e32-fae6-4026-8b91-a9b90754d43b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:57.591527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:57.592326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.592610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.592888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.599978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:57.601483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:18:57.618343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:57.619857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:18:57.621331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:57.624894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 525/1043] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:57.626676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:57.701642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:57.734184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32be71-5823-4b9a-a517-e438f22a87a3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:57.771411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:57.772078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.772361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.772631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.773443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:57.773745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:57.775172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40813f4-efa6-4b44-81a0-92a20ee321d7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:57.793086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 519/1044] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:57.794600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:57.803673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:57.898818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929683c0-42ec-4dcb-b5ff-a9f6d5203e56 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:57.931065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c8375-b1ce-41a6-a10a-6cc1f0640d2b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.931419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c8375-b1ce-41a6-a10a-6cc1f0640d2b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.931581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c8375-b1ce-41a6-a10a-6cc1f0640d2b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.932974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 520/1045] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:18:57.938683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.938977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.939255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.941639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:57.942298 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:57.942590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:57.942855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:57.943696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] RBAC: Adding query filter params (role.id=0bb38eca871c4a7f9e6e24200e62626a, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=862f9782f8574b8bb99d38c51333277f) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:57.944007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:57.945469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:57.952038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 526/1046] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:57 2022] GET /identity/v3/role_assignments?user.id=862f9782f8574b8bb99d38c51333277f&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0bb38eca871c4a7f9e6e24200e62626a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:57.968319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:57.969609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:58.052951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 521/1047] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:58.076344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:58.152869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:58.185907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b6e8a-047c-443c-841a-9691528c440b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:58.220024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:58.220742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:58.221024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:58.221299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:58.231593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:18:58.240639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:58.241915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:18:58.242800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:58.245514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 527/1048] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:58.246538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:58.308093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:58.352108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5724898-a9b8-4e0b-b0d9-cfec510ff564 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:58.386707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:58.387426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:58.387732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:58.387966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:58.388711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:58.389019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:58.390386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:58.399194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 522/1049] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:58.405142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:58.509236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a014af-0ded-4597-89c2-0e00e18192d1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:58.537281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:58.545027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:58.545950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:58.546235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:58.546505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] PATH_INFO: `/v3/users/0ba5a05105334b108c44c9597aa9fe48` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:58.560374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba5a05105334b108c44c9597aa9fe48)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:58.561862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:58.563708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 528/1050] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:18:58 2022] GET /identity/v3/users/0ba5a05105334b108c44c9597aa9fe48 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:58.569349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:58.670989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d385ef4-b304-497f-b0bf-7bf65eaa4053 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:58.694292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:58.707549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:58.708193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:58.708471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:58.708738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:58.775251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:58.776603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:58.872449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 523/1051] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:18:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:18:58.878328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:58.985188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56319-f9c3-4db7-b72a-25a1e0110537 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:59.020599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ce138-cd84-4b89-a982-1819f664e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:59.021506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ce138-cd84-4b89-a982-1819f664e70c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.021786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ce138-cd84-4b89-a982-1819f664e70c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.022059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ce138-cd84-4b89-a982-1819f664e70c None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.023259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ce138-cd84-4b89-a982-1819f664e70c None admin] RBAC: Authorizing `identity:get_project(project_id=2051ec1bdff0407284a22c12e374295a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:59.024677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ce138-cd84-4b89-a982-1819f664e70c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:59.026607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 529/1052] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:18:58 2022] GET /identity/v3/projects/2051ec1bdff0407284a22c12e374295a => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:59.032748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:59.059939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:59.092995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535a45-4e4b-49c1-be45-7afabfd626e4 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.093335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535a45-4e4b-49c1-be45-7afabfd626e4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.093585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d535a45-4e4b-49c1-be45-7afabfd626e4 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.095113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 530/1053] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:18:59.100447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.100734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.101000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.118739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:18:59.119928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:18:59.150590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d6225-4e3a-428d-9216-77e81b839548 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:59.173262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:59.192256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 531/1054] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:18:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:18:59.198344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:59.199201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.199831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.200606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.201786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=2051ec1bdff0407284a22c12e374295a, user.id=0ba5a05105334b108c44c9597aa9fe48) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:59.202238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:59.203816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:59.209787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 524/1055] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:18:59 2022] GET /identity/v3/role_assignments?user.id=0ba5a05105334b108c44c9597aa9fe48&scope.project.id=2051ec1bdff0407284a22c12e374295a&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:59.215222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:59.339324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147d22b-3bc5-4d2d-8552-6d3ced89de76 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:59.376741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:59.377569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.377830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.378109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.378768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:59.380161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:18:59.392572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:18:59.393688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 13:18:59.394613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:18:59.397741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 532/1056] 10.222.0.24 () {56 vars in 1105 bytes} [Thu May 19 13:18:59 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:18:59.399903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:59.490137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:59.507235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73c947-d759-44f1-a4ae-a127d6496e59 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:59.543760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:59.544465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.544747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.545021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.545773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] RBAC: Adding query filter params (name=image) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:59.546077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:59.547431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:59.552782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 525/1057] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:18:59.558413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:59.611404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:18:59.664682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6689bf5-bfbd-48f1-9976-f6758b26f4e5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:59.700130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:59.700795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.701082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.701335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.702146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] RBAC: Adding query filter params (type=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:18:59.702456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:59.703835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401524e-2c4d-4eb5-94b8-f7fe24deccea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:59.709208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 533/1058] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:18:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:18:59.714778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:18:59.814388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56065af-33da-46ff-bf44-937836e6db9b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:18:59.847695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a36920-0b1e-47e0-ab97-65e44f7ae51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:18:59.848346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36920-0b1e-47e0-ab97-65e44f7ae51b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:18:59.848627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36920-0b1e-47e0-ab97-65e44f7ae51b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:18:59.848900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36920-0b1e-47e0-ab97-65e44f7ae51b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:18:59.849684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36920-0b1e-47e0-ab97-65e44f7ae51b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:18:59.851042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36920-0b1e-47e0-ab97-65e44f7ae51b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:18:59.870323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 526/1059] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:18:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:19:00.140672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1181f4-7c39-4abf-b2b0-12361b3da2d7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:00.140875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1181f4-7c39-4abf-b2b0-12361b3da2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:00.141139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1181f4-7c39-4abf-b2b0-12361b3da2d7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:00.142544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 534/1060] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:00.147891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:00.148181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:00.148462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:00.159540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:00.166491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:00.168482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:00.245833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 527/1061] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:00.270180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1181f4-7c39-4abf-b2b0-12361b3da2d7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:00.379198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1181f4-7c39-4abf-b2b0-12361b3da2d7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:00.394143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:00.416386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:00.417199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:00.417506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:00.417794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:00.428532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:00.436698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:00.437923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:19:00.438878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:00.441791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 535/1062] 10.222.0.24 () {56 vars in 1120 bytes} [Thu May 19 13:19:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:00.443888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:00.548998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010f0e-f579-4b3b-b395-56f6e099a43a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:00.578765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:00.584350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:00.585024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:00.585293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:00.585603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:00.586369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:00.586674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:00.588032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:00.596855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 528/1063] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:19:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:00.602525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:00.703255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e066c3d-ace0-40fc-a780-03845e376034 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:00.737085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:00.737913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:00.738193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:00.738469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:00.749274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:00.750795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:19:00.767867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:00.768828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:00.769807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:19:00.769807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:19:00.769807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:00.769807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:00.773768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 536/1064] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:00.775017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:00.881121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe983-a320-4db3-834f-51a43d61256d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:00.915995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:00.916661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:00.916941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:00.917216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:00.917955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:00.918348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:00.919844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:00.931939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 529/1065] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:00.937663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:00.966958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.047452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a61efa-9069-47d8-951b-f631b3290dab None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.082386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.083179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.083463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.083740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.092449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.093988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:19:01.108293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:01.109297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:01.115250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 537/1066] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:19:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:01.115869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.175041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.222198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2124b0-05f7-4875-a6c8-e45b8716383f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.267595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.268285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.268566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.268838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.269737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:01.270049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.271504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4640caa-bd7c-479c-9252-558781dcd67b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.291206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 530/1067] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:01.296914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.388267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.405294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe3eac-ec45-41ab-9326-35d831618c27 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.431540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dea4d7-b251-4b46-af55-5ed133bd60cb None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.432113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dea4d7-b251-4b46-af55-5ed133bd60cb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.432395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dea4d7-b251-4b46-af55-5ed133bd60cb None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.433878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 531/1068] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:01.439595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.439881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.440156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.443657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.444626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.445352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.445800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.457095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:01.458306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:01.460007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.461540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:19:01.479760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:01.480719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:01.482552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:19:01.482552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:19:01.482552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:01.482552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:01.482552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 538/1069] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:01.486519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.531018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 532/1070] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:01.553645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.590377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1947f9-a3af-4e72-9824-e80376b749f1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.596231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.627836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.628508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.628790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.629066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.629820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:01.630125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.631518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.638881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.649109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 539/1071] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:01.650064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.670864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec1d76-cb2b-484d-8e47-f159a9f3457c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.707452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.708354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.708632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.708907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.720181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:01.728216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:01.729732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:01.731103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 533/1072] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:19:01.735308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.755146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16dbd9-58eb-4e84-bc33-3ca67829aedb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.791163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.791965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.792250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.792527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.799414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.800873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:19:01.814559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:01.815558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:01.816663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 540/1073] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:01.821404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.848628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9da2b-b9c6-4632-b065-0e78f21311f0 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.850762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.886450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.887582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.888057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.888544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.889631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:01.889946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.891360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.893172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:01.905228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 534/1074] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:01.906347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:01.942319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26115c7f-13ec-4298-88e3-3baad13dd7d2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:01.979500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:01.980203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:01.980462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:01.980832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:01.981902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:01.982157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:01.983658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:01.997025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 541/1075] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:02.003239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:02.024138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aa6db-35bf-482f-ae5a-888154958e76 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.059095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.060080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.060361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.060748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] PATH_INFO: `/v3/users/0ba5a05105334b108c44c9597aa9fe48` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.061982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba5a05105334b108c44c9597aa9fe48)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.063628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.065387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 535/1076] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:01 2022] GET /identity/v3/users/0ba5a05105334b108c44c9597aa9fe48 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:02.070913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:02.103150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:02.112134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1932406-bb82-43a8-a9fb-b314e67b6a14 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.131320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:02.151683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5c9ee-306e-4b37-af60-f55d9b9034f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.152671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c9ee-306e-4b37-af60-f55d9b9034f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.153028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c9ee-306e-4b37-af60-f55d9b9034f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.153288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5c9ee-306e-4b37-af60-f55d9b9034f5 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/862f9782f8574b8bb99d38c51333277f/roles/0bb38eca871c4a7f9e6e24200e62626a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.155549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c9ee-306e-4b37-af60-f55d9b9034f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=862f9782f8574b8bb99d38c51333277f, role_id=0bb38eca871c4a7f9e6e24200e62626a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.156979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5c9ee-306e-4b37-af60-f55d9b9034f5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.170304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 542/1077] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:19:02 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/862f9782f8574b8bb99d38c51333277f/roles/0bb38eca871c4a7f9e6e24200e62626a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:19:02.171542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07e9c-af08-4621-97f2-0081ba48cd2d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.171824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07e9c-af08-4621-97f2-0081ba48cd2d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.172097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af07e9c-af08-4621-97f2-0081ba48cd2d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.173348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 543/1078] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:02.179053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.179413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.179795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.195241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754735d-777d-4279-9c8a-e8d79663aa54 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.197106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:02.198362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:02.241848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.243866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.244202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.244479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.246214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=2051ec1bdff0407284a22c12e374295a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.248038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.254819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 536/1079] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:02 2022] GET /identity/v3/projects/2051ec1bdff0407284a22c12e374295a => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:02.258684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:02.273494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 544/1080] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:19:02.296301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:02.384188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:02.388032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45b118-120d-4e8c-b13c-c7f958da1aa4 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.416125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b57a1c-fb7b-4a51-9f28-38794a29856e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.424252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.425144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.425433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.425716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a/users/0ba5a05105334b108c44c9597aa9fe48/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.426821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:02.427688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] RBAC: Authorizing `identity:create_grant(project_id=2051ec1bdff0407284a22c12e374295a, user_id=0ba5a05105334b108c44c9597aa9fe48, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.429118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.446642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 537/1081] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:19:02 2022] PUT /identity/v3/projects/2051ec1bdff0407284a22c12e374295a/users/0ba5a05105334b108c44c9597aa9fe48/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:19:02.455372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.456045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.456330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.456593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.534356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.535748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.647724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 545/1082] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:19:02.653917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:02.761489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980922f-9ab3-47ef-9a4d-c0d50c687a79 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.797749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.798440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.798721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.798986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.810115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.811560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-eb9eca87-7037-446e-bd48-3052a4d75611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:19:02.828667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:02.829743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:02.830769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:19:02.830769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:19:02.830769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:02.830769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:02.830769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 538/1083] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:02.837105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:02.847565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:02.941132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ed436-74ab-42f3-82f0-cd924c2c42ff None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:02.977046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:02.977735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:02.978021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:02.978296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:02.979016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:02.979327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:02.980735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626027dd-e26e-4ec6-8b26-2a9dff75e7aa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:02.992722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 546/1084] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:04.370356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb04d0-1f51-4e89-b0e5-e37ed969256c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.370928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb04d0-1f51-4e89-b0e5-e37ed969256c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.370928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb04d0-1f51-4e89-b0e5-e37ed969256c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.372439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 539/1085] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:04.378194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f79a49-cf33-4aa9-b770-e7727b446cc1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.378639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f79a49-cf33-4aa9-b770-e7727b446cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.378855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f79a49-cf33-4aa9-b770-e7727b446cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.396139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-87f79a49-cf33-4aa9-b770-e7727b446cc1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:04.397307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-87f79a49-cf33-4aa9-b770-e7727b446cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:04.460845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:04.467386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 547/1086] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:19:04.490891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb04d0-1f51-4e89-b0e5-e37ed969256c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:04.508498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:04.603282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb04d0-1f51-4e89-b0e5-e37ed969256c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:04.638227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:04.639031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.639314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.639592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.650795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:04.654513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7107bff-1126-42e3-a9f3-e0cfa3d09a94 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.654793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7107bff-1126-42e3-a9f3-e0cfa3d09a94 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.655065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7107bff-1126-42e3-a9f3-e0cfa3d09a94 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.656403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 548/1087] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:04.659578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:04.660649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:04.661591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 540/1088] 10.222.0.24 () {56 vars in 1120 bytes} [Thu May 19 13:19:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:04.662366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67781-fa63-4932-b457-d705639aa07e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.662454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67781-fa63-4932-b457-d705639aa07e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.662760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e67781-fa63-4932-b457-d705639aa07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.666944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:04.681169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-81e67781-fa63-4932-b457-d705639aa07e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:04.682405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-81e67781-fa63-4932-b457-d705639aa07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:04.738593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:04.750039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 549/1089] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:04.770111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2b00-3750-4e63-a617-450e00b878f9 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:04.774086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e67781-fa63-4932-b457-d705639aa07e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:04.804681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:04.805349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.805661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.805939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.806687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:04.806996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:04.808358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:04.816898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 541/1090] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:19:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:04.822444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:04.872985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e67781-fa63-4932-b457-d705639aa07e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:04.898577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:04.907034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:04.908013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.908296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.908571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.919071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:04.921844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729671e-6deb-4781-84f0-4bf519a9fa9f None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:04.927126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:04.928278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:04.929148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 550/1091] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:04.934300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:04.954669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:04.955455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:04.955732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:04.955999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:04.957892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:04.967290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:04.968742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:19:04.985626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:04.986741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:04.987702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:19:04.987702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:19:04.987702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:04.987702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:04.987702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 542/1092] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:19:04.992609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.034563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc899d-f322-4f01-a008-b78063f549a2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.068595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.069247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.069535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.069826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.070600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:05.070899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.072246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.080312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 551/1093] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:05.086014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.104788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c0263-92d1-438f-8b12-ad822a5884e5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.137224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.139624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.140210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.140709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.140822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.141504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:05.141802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.143159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.154955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 543/1094] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:05.160534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.189668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e99561-ec95-40e9-a076-a363355a7e2e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.207306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.224600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.225470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.225755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.226029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] PATH_INFO: `/v3/users/0ba5a05105334b108c44c9597aa9fe48` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.227240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba5a05105334b108c44c9597aa9fe48)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.228732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.230386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 552/1095] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/users/0ba5a05105334b108c44c9597aa9fe48 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:05.236209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.274189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1875793-8e14-4c93-95fd-823c66ff7e65 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.307467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.308277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.308556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.308913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.316214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.317682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:19:05.331734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:05.332805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:05.333860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 544/1096] 10.222.0.24 () {56 vars in 1093 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:19:05.338809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.340692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607678c-b18e-461e-a8c7-02592f35ff61 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.358717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.395999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.396668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.396919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.397193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.438908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b40f2-1e5c-4ae8-b86d-8961d3568c73 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.446237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.471444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.472913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.474740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.475339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.475594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.475864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.476709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:05.477018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.478458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.495893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 545/1097] 10.222.0.24 () {56 vars in 1133 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:05.502019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.577561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 553/1098] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:19:05.578723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.597592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.620066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c27cec-c53f-4d39-b859-0ef8a6de7f45 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.654683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.655258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.655541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.655818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.679762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e178f-7389-4955-88a0-2f83212d48a4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.685831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.715496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.716348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.716622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.716892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.718090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] RBAC: Authorizing `identity:get_project(project_id=2051ec1bdff0407284a22c12e374295a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.719521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.726330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 554/1099] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/projects/2051ec1bdff0407284a22c12e374295a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:05.726494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.726494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.727400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.809932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.829008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 546/1100] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:19:05.830104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:05.831417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e768bc-d002-4969-afa0-3c70b49d7c16 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.867211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.867871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.868132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.868403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.869228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=2051ec1bdff0407284a22c12e374295a, user.id=0ba5a05105334b108c44c9597aa9fe48) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:05.869569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.871001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:05.876924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 555/1101] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/role_assignments?user.id=0ba5a05105334b108c44c9597aa9fe48&scope.project.id=2051ec1bdff0407284a22c12e374295a&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:05.890328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:05.931976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f76833-5374-4a58-93a1-03ad4b4d9658 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:05.970507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:05.971307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:05.971587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:05.971857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:05.984061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:05.985563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:19:06.004576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:06.005687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:06.006640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:19:06.006640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:19:06.006640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:19:06.006640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:06.006640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 547/1102] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:19:06.011927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:06.120415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1cc69-c4d7-4568-a46d-0ed0dcc6f464 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:06.153317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:06.154006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:06.154289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:06.154562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:06.155281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:06.155586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:06.156962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:06.169274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 556/1103] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:06.175021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:06.280021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724a373-1677-44a7-b768-83bd38cad0fe None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:06.297676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:06.316076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:06.316869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:06.317148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:06.317434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:06.324150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:06.325926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:19:06.339334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:06.340297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:06.341230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 548/1104] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:06.346530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:06.359460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:06.447214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f53e64-7498-454d-93ba-3ef0ecef7aa4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:06.481858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:06.482509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:06.482790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:06.483066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:06.483845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:06.484153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:06.485619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e178-d17a-4358-a522-b74a6fcae3f1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:06.497950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 557/1105] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:06.504043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:06.609557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5222891-ee1f-4135-ab0f-33d3b187da9c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:06.643934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:06.644580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:06.644856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:06.645128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:06.645996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] RBAC: Adding query filter params (role.id=0bb38eca871c4a7f9e6e24200e62626a, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=862f9782f8574b8bb99d38c51333277f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:06.646305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:06.647726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669d192-32c5-4563-8683-eddacab6d0a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:06.654170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 549/1106] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:06 2022] GET /identity/v3/role_assignments?user.id=862f9782f8574b8bb99d38c51333277f&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0bb38eca871c4a7f9e6e24200e62626a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:08.148978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9fd65-dc5a-4857-9e1c-f6c251678bd5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:08.149948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9fd65-dc5a-4857-9e1c-f6c251678bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:08.149948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9fd65-dc5a-4857-9e1c-f6c251678bd5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:08.152067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 558/1107] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:08.160180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:08.160615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:08.160723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:08.186016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:08.187215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:08.227796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:08.267841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 550/1108] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:19:08.293299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9fd65-dc5a-4857-9e1c-f6c251678bd5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:08.376382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:08.423951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9fd65-dc5a-4857-9e1c-f6c251678bd5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:08.460730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccec918-a4ec-4c01-94aa-7bf7056172f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:08.461535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccec918-a4ec-4c01-94aa-7bf7056172f1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:08.461816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccec918-a4ec-4c01-94aa-7bf7056172f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:08.462088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccec918-a4ec-4c01-94aa-7bf7056172f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:08.463247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccec918-a4ec-4c01-94aa-7bf7056172f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:08.464692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccec918-a4ec-4c01-94aa-7bf7056172f1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:08.466437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 559/1109] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:08.475162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:08.587390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301724e5-fe44-4b87-9189-7de50f11c94f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:08.645452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb015e9-f4d1-4244-a3f3-5f71018c1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:08.646134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb015e9-f4d1-4244-a3f3-5f71018c1f0c None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:08.646416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb015e9-f4d1-4244-a3f3-5f71018c1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:08.646688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb015e9-f4d1-4244-a3f3-5f71018c1f0c None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:08.647387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb015e9-f4d1-4244-a3f3-5f71018c1f0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:08.648809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb015e9-f4d1-4244-a3f3-5f71018c1f0c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:08.670733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 551/1110] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:19:08 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:19:10.903519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f90097-1356-401e-bf89-1c69e2d8e1ff None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:10.904406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f90097-1356-401e-bf89-1c69e2d8e1ff None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:10.904406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f90097-1356-401e-bf89-1c69e2d8e1ff None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:10.906626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 560/1111] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:10.914952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:10.915439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:10.915535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:10.933397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:10.934786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:10.954980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:11.008653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 552/1112] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:19:11.033658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f90097-1356-401e-bf89-1c69e2d8e1ff None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:11.155230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f90097-1356-401e-bf89-1c69e2d8e1ff None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:11.192556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:11.193404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:11.193720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:11.193993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] PATH_INFO: `/v3/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:11.195510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] RBAC: Authorizing `identity:get_role(role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:11.196888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:11.198354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 561/1113] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:11 2022] GET /identity/v3/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:11.204436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:11.311361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde815cc-5d1d-4ae2-82cd-8e28e87fdb3d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:11.350024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:11.350882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:11.351165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:11.351435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] PATH_INFO: `/v3/users/9bba7e9450844f349a7d12808fa16fe5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:11.364936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] RBAC: Authorizing `identity:get_user(user_id=9bba7e9450844f349a7d12808fa16fe5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:11.366397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:11.368051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 553/1114] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:11 2022] GET /identity/v3/users/9bba7e9450844f349a7d12808fa16fe5 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:11.374681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:11.388285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:11.510473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389e4d9-1cc4-4e8a-ab84-71505724550b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:11.548011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:11.548649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:11.548931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:11.549205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:11.617682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:11.619134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:11.627240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:11.721514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 562/1115] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:19:11.727839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:11.743875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:11.829966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68a72c-f9c0-480c-8537-0791d4b31b03 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:11.864978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ae36d-d1df-4dbb-a26f-66e317bb7200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:11.865857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ae36d-d1df-4dbb-a26f-66e317bb7200 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:11.866136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ae36d-d1df-4dbb-a26f-66e317bb7200 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:11.866409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ae36d-d1df-4dbb-a26f-66e317bb7200 None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:11.867639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ae36d-d1df-4dbb-a26f-66e317bb7200 None admin] RBAC: Authorizing `identity:get_project(project_id=2051ec1bdff0407284a22c12e374295a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:11.869052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ae36d-d1df-4dbb-a26f-66e317bb7200 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:11.870786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 554/1116] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:11 2022] GET /identity/v3/projects/2051ec1bdff0407284a22c12e374295a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:19:11.878009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:11.995437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6ebf4-d5c1-43df-b4b5-d09a219a44b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:12.003041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:12.032163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:12.032810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:12.033092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:12.033370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:12.034241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] RBAC: Adding query filter params (role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=2051ec1bdff0407284a22c12e374295a, user.id=9bba7e9450844f349a7d12808fa16fe5) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:12.034562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:12.035994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e42d9f-29d5-464c-8a55-14c1fe5caf59 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:12.041950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 563/1117] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:11 2022] GET /identity/v3/role_assignments?user.id=9bba7e9450844f349a7d12808fa16fe5&scope.project.id=2051ec1bdff0407284a22c12e374295a&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:14.244343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89308-ad2e-4104-9e5f-e169312aeaf7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:14.244921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89308-ad2e-4104-9e5f-e169312aeaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:14.244921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa89308-ad2e-4104-9e5f-e169312aeaf7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:14.246396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 555/1118] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:14.253136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:14.253907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:14.254059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:14.273887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:14.275082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:14.345435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 564/1119] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:14.369173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89308-ad2e-4104-9e5f-e169312aeaf7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:14.458921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:14.483106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa89308-ad2e-4104-9e5f-e169312aeaf7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:14.519415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:14.520201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:14.520488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:14.520760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] PATH_INFO: `/v3/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:14.522366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] RBAC: Authorizing `identity:get_role(role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:14.523742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:14.525131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 556/1120] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:14 2022] GET /identity/v3/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:14.535165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:14.638322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:14.642033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08a5c-3485-49ac-bbab-01f608b57d01 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:14.680852 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:14.681739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:14.682022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:14.682301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] PATH_INFO: `/v3/users/9bba7e9450844f349a7d12808fa16fe5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:14.683549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=9bba7e9450844f349a7d12808fa16fe5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:14.685073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:14.686752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 565/1121] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:14 2022] GET /identity/v3/users/9bba7e9450844f349a7d12808fa16fe5 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:14.693306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:14.799900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96afe20-cee3-40b3-a958-4c92f0c6ad18 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:14.834435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e67ae1-334b-479b-8c9d-cd6cf40d92ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:14.835304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67ae1-334b-479b-8c9d-cd6cf40d92ff None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:14.835584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67ae1-334b-479b-8c9d-cd6cf40d92ff None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:14.835860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67ae1-334b-479b-8c9d-cd6cf40d92ff None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:14.837109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e67ae1-334b-479b-8c9d-cd6cf40d92ff None admin] RBAC: Authorizing `identity:get_project(project_id=2051ec1bdff0407284a22c12e374295a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:14.838707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e67ae1-334b-479b-8c9d-cd6cf40d92ff None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:14.840802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 557/1122] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:14 2022] GET /identity/v3/projects/2051ec1bdff0407284a22c12e374295a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:14.846859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:14.944773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94e932-91be-4c79-98c5-2d5385962bb7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:14.979900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff86fcd-b7fd-4ea8-b731-e4598b01681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:14.980721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff86fcd-b7fd-4ea8-b731-e4598b01681c None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:14.980994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff86fcd-b7fd-4ea8-b731-e4598b01681c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:14.981266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff86fcd-b7fd-4ea8-b731-e4598b01681c None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a/users/9bba7e9450844f349a7d12808fa16fe5/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:14.983866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff86fcd-b7fd-4ea8-b731-e4598b01681c None admin] RBAC: Authorizing `identity:create_grant(project_id=2051ec1bdff0407284a22c12e374295a, user_id=9bba7e9450844f349a7d12808fa16fe5, role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:14.985296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff86fcd-b7fd-4ea8-b731-e4598b01681c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:15.002055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 566/1123] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:19:14 2022] PUT /identity/v3/projects/2051ec1bdff0407284a22c12e374295a/users/9bba7e9450844f349a7d12808fa16fe5/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:19:17.170919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840e27-454a-42ba-b05e-0e4943094cf2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:17.171506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840e27-454a-42ba-b05e-0e4943094cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:17.171506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840e27-454a-42ba-b05e-0e4943094cf2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:17.172859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 558/1124] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:17.179278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:17.180023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:17.180166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:17.201527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:17.202739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:17.226126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:17.277527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 567/1125] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:17.301133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08840e27-454a-42ba-b05e-0e4943094cf2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:17.322424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:17.427359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08840e27-454a-42ba-b05e-0e4943094cf2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:17.464381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:17.465788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:17.466083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:17.466361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] PATH_INFO: `/v3/roles/5ca7949e3dfa46bb8955287ebf4caf67` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:17.467992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] RBAC: Authorizing `identity:get_role(role_id=5ca7949e3dfa46bb8955287ebf4caf67)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:17.469359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:17.470783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 559/1126] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:17 2022] GET /identity/v3/roles/5ca7949e3dfa46bb8955287ebf4caf67 => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:17.477652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:17.589516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a952154-3ed7-495d-9238-7322f2bf842b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:17.628936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:17.629829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:17.630110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:17.630384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] PATH_INFO: `/v3/users/9bba7e9450844f349a7d12808fa16fe5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:17.631642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] RBAC: Authorizing `identity:get_user(user_id=9bba7e9450844f349a7d12808fa16fe5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:17.633076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:17.634769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 568/1127] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:17 2022] GET /identity/v3/users/9bba7e9450844f349a7d12808fa16fe5 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:17.641607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:17.737973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:17.753842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594040-613d-4bd9-a40c-7561526d14e5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:17.790876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:17.791514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:17.791794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:17.792067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:17.863702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:17.865148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:17.951094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:17.969711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 560/1128] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:19:17.975803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:18.007539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:18.084940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16796b21-b55a-4be6-bf41-4c561edfa180 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:18.123211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5eb57-84ce-4874-b2e8-fa9cd7deb849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:18.124078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5eb57-84ce-4874-b2e8-fa9cd7deb849 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:18.124364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5eb57-84ce-4874-b2e8-fa9cd7deb849 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:18.124641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5eb57-84ce-4874-b2e8-fa9cd7deb849 None admin] PATH_INFO: `/v3/projects/2051ec1bdff0407284a22c12e374295a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:18.125887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5eb57-84ce-4874-b2e8-fa9cd7deb849 None admin] RBAC: Authorizing `identity:get_project(project_id=2051ec1bdff0407284a22c12e374295a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:18.127328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5eb57-84ce-4874-b2e8-fa9cd7deb849 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:18.129008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 569/1129] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:17 2022] GET /identity/v3/projects/2051ec1bdff0407284a22c12e374295a => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:18.135215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:18.239639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba20b8-2b78-4357-bb1b-30df131ac419 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:18.282223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:18.282880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:18.283158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:18.283434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:18.284274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] RBAC: Adding query filter params (role.id=5ca7949e3dfa46bb8955287ebf4caf67, scope.project.id=2051ec1bdff0407284a22c12e374295a, user.id=9bba7e9450844f349a7d12808fa16fe5) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:18.284583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:18.286052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bec312-0627-4215-b7bc-ce8a3606e3a3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:18.292306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 561/1130] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:18 2022] GET /identity/v3/role_assignments?user.id=9bba7e9450844f349a7d12808fa16fe5&scope.project.id=2051ec1bdff0407284a22c12e374295a&role.id=5ca7949e3dfa46bb8955287ebf4caf67 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:20.514465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aceaff-037d-4a10-a834-2b90da261765 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:20.515305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aceaff-037d-4a10-a834-2b90da261765 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:20.515442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aceaff-037d-4a10-a834-2b90da261765 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:20.518966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 570/1131] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:20.526026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:20.526753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:20.526944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:20.552754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:20.554579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:20.615966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:20.642088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 562/1132] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:20.675719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aceaff-037d-4a10-a834-2b90da261765 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:20.777147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:20.788243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aceaff-037d-4a10-a834-2b90da261765 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:20.827909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e097d-1b48-480b-ad2b-165f3d1f7c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:20.828687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e097d-1b48-480b-ad2b-165f3d1f7c56 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:20.828963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e097d-1b48-480b-ad2b-165f3d1f7c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:20.829237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e097d-1b48-480b-ad2b-165f3d1f7c56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:20.830658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e097d-1b48-480b-ad2b-165f3d1f7c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:20.832091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e097d-1b48-480b-ad2b-165f3d1f7c56 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:20.833749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 571/1133] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:20.842024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:20.957460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f2fb5-3401-40c4-8098-b6c387eb21c3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:20.994004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60d066-351a-41c5-97b6-0e5adbed0cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:20.994646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60d066-351a-41c5-97b6-0e5adbed0cca None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:20.994923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60d066-351a-41c5-97b6-0e5adbed0cca None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:20.995194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60d066-351a-41c5-97b6-0e5adbed0cca None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:20.995895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60d066-351a-41c5-97b6-0e5adbed0cca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:20.997459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60d066-351a-41c5-97b6-0e5adbed0cca None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:21.024363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 563/1134] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:19:20 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:19:23.229685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d2a57-da79-4162-ab0b-26160a56bdbb None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:23.230236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d2a57-da79-4162-ab0b-26160a56bdbb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:23.230236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d2a57-da79-4162-ab0b-26160a56bdbb None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:23.232706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 572/1135] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:23.241454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:23.241850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:23.242035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:23.261501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:23.262721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:23.289904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:23.334755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 564/1136] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:23.372094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d2a57-da79-4162-ab0b-26160a56bdbb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:23.493895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d2a57-da79-4162-ab0b-26160a56bdbb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:23.529719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67574216-56ec-4d65-8d0c-d3538daafe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:23.530501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67574216-56ec-4d65-8d0c-d3538daafe33 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:23.530783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67574216-56ec-4d65-8d0c-d3538daafe33 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:23.531030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67574216-56ec-4d65-8d0c-d3538daafe33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:23.532349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67574216-56ec-4d65-8d0c-d3538daafe33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:23.533822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67574216-56ec-4d65-8d0c-d3538daafe33 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:23.535413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 573/1137] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:19:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:23.542355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:23.646720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a988fc-d5d6-4707-b7a0-749ffb1e1b8b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:23.682167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb854e-fded-482e-a2e8-0dbb8224fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:23.682822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb854e-fded-482e-a2e8-0dbb8224fdf4 None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:23.683093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb854e-fded-482e-a2e8-0dbb8224fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:23.683364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb854e-fded-482e-a2e8-0dbb8224fdf4 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:23.684210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb854e-fded-482e-a2e8-0dbb8224fdf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:23.685663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb854e-fded-482e-a2e8-0dbb8224fdf4 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:23.708167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 565/1138] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:19:23 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:19:25.966410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b510089-728e-4e01-800d-6003b72c5482 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:25.967547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b510089-728e-4e01-800d-6003b72c5482 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:25.967547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b510089-728e-4e01-800d-6003b72c5482 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:25.969512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 574/1139] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:25.978510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:25.979275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:25.979433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:25.998243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:25.999436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:26.013453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:26.075114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 566/1140] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:26.104370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b510089-728e-4e01-800d-6003b72c5482 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:26.139317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:26.218730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b510089-728e-4e01-800d-6003b72c5482 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:26.255090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:26.255983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:26.256328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:26.256606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:26.267695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:26.275245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:26.276360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:26.277609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 575/1141] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:26.285214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:26.400235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d264c-385d-4803-9c6d-3198c18f2f3e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:26.435241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:26.435894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:26.436175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:26.436433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:26.437178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:26.437511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:26.438882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:26.446844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 567/1142] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:26.455449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:26.564931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:26.576527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ba55-b2a9-4bef-8d24-2e4074869912 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:26.614872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:26.615745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:26.616026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:26.616303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] PATH_INFO: `/v3/users/a5b3ca9b45d54d1ca88b1d1313ca3d03` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:26.631151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] RBAC: Authorizing `identity:get_user(user_id=a5b3ca9b45d54d1ca88b1d1313ca3d03)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:26.632612 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:26.634381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 576/1143] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:26 2022] GET /identity/v3/users/a5b3ca9b45d54d1ca88b1d1313ca3d03 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:26.642038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:26.663324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:26.759654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1046c0b-d387-45e0-97fb-c550e1ef9d0b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:26.794355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:26.794983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:26.795257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:26.795533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:26.873875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:26.876355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:26.892062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:26.994623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 568/1144] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:19:27.000934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:27.108463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec226e2-635a-48ed-8c2a-9e8c696fdf0c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:27.145710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b74e5-0f80-4ad3-b33e-d8f7ce3da365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:27.146489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b74e5-0f80-4ad3-b33e-d8f7ce3da365 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:27.146774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b74e5-0f80-4ad3-b33e-d8f7ce3da365 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:27.147047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b74e5-0f80-4ad3-b33e-d8f7ce3da365 None admin] PATH_INFO: `/v3/projects/be6fab1fef114a6b8040bc07420d8587` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:27.148350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b74e5-0f80-4ad3-b33e-d8f7ce3da365 None admin] RBAC: Authorizing `identity:get_project(project_id=be6fab1fef114a6b8040bc07420d8587)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:27.149843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b74e5-0f80-4ad3-b33e-d8f7ce3da365 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:27.151524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 577/1145] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:26 2022] GET /identity/v3/projects/be6fab1fef114a6b8040bc07420d8587 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:27.161494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:27.275323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b31f5-1084-4ced-8f20-be0c0a98f611 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:27.288096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:27.321156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:27.321851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:27.322129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:27.322401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:27.323231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=be6fab1fef114a6b8040bc07420d8587, user.id=a5b3ca9b45d54d1ca88b1d1313ca3d03) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:27.323539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:27.325169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e060b6c-cb7c-485c-98fb-ac930da65042 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:27.331021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 569/1146] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:27 2022] GET /identity/v3/role_assignments?user.id=a5b3ca9b45d54d1ca88b1d1313ca3d03&scope.project.id=be6fab1fef114a6b8040bc07420d8587&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:29.568558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55127f6a-12a5-4d4b-8e83-df52ff7d984c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:29.569404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55127f6a-12a5-4d4b-8e83-df52ff7d984c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:29.569404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55127f6a-12a5-4d4b-8e83-df52ff7d984c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:29.571487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 578/1147] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:29.578408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:29.579079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:29.579197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:29.605886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:29.607448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:29.680536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 570/1148] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:29.706153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55127f6a-12a5-4d4b-8e83-df52ff7d984c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:29.725474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:29.816951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55127f6a-12a5-4d4b-8e83-df52ff7d984c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:29.851357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:29.852073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:29.852351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:29.852627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:29.862667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:29.870252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:29.871449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:29.872517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 579/1149] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:29.880087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:29.984373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab9ff8-5bf3-4833-ba06-ec7e7b8bc2cf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:29.986740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:30.020884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:30.021571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:30.021856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:30.022137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:30.022885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:30.023192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:30.024599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:30.032804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 571/1150] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:30.038961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:30.119212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:30.146783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65e099-2a71-42fc-b6ce-4f3d7bf5c87c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:30.182538 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:30.183395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:30.183761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:30.184045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] PATH_INFO: `/v3/users/a5b3ca9b45d54d1ca88b1d1313ca3d03` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:30.185360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] RBAC: Authorizing `identity:get_user(user_id=a5b3ca9b45d54d1ca88b1d1313ca3d03)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:30.186837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:30.188482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 580/1151] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:30 2022] GET /identity/v3/users/a5b3ca9b45d54d1ca88b1d1313ca3d03 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:30.194693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:30.299816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfed1c-f017-4505-9e46-22d0af83303d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:30.335799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e65f2-0081-4c7f-be79-79297641c5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:30.336664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e65f2-0081-4c7f-be79-79297641c5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:30.336950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e65f2-0081-4c7f-be79-79297641c5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:30.337229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e65f2-0081-4c7f-be79-79297641c5d8 None admin] PATH_INFO: `/v3/projects/be6fab1fef114a6b8040bc07420d8587` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:30.338507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e65f2-0081-4c7f-be79-79297641c5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=be6fab1fef114a6b8040bc07420d8587)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:30.340436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e65f2-0081-4c7f-be79-79297641c5d8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:30.342242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 572/1152] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:30 2022] GET /identity/v3/projects/be6fab1fef114a6b8040bc07420d8587 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:30.348434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:30.462334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8042a1-e377-45f5-a690-1d83b65ef608 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:30.497936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c344e-e026-49e7-b81b-183e4ba52c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:30.498845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c344e-e026-49e7-b81b-183e4ba52c5b None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:30.499126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c344e-e026-49e7-b81b-183e4ba52c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:30.499408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c344e-e026-49e7-b81b-183e4ba52c5b None admin] PATH_INFO: `/v3/projects/be6fab1fef114a6b8040bc07420d8587/users/a5b3ca9b45d54d1ca88b1d1313ca3d03/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:30.501643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c344e-e026-49e7-b81b-183e4ba52c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=be6fab1fef114a6b8040bc07420d8587, user_id=a5b3ca9b45d54d1ca88b1d1313ca3d03, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:30.503088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c344e-e026-49e7-b81b-183e4ba52c5b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:30.520127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 581/1153] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:19:30 2022] PUT /identity/v3/projects/be6fab1fef114a6b8040bc07420d8587/users/a5b3ca9b45d54d1ca88b1d1313ca3d03/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:19:32.743714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65dcb-9151-4f1d-b175-a72d3c106bd1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:32.744355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65dcb-9151-4f1d-b175-a72d3c106bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:32.744355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65dcb-9151-4f1d-b175-a72d3c106bd1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:32.745693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 573/1154] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:32.752658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:32.753368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:32.753582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:32.763259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:32.775262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:32.776458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:32.863738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 582/1155] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:32.888101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65dcb-9151-4f1d-b175-a72d3c106bd1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:32.928032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:33.002723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65dcb-9151-4f1d-b175-a72d3c106bd1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:33.042885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:33.043667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:33.043952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:33.044224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:33.056791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:33.064401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:33.065644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:33.066874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 574/1156] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:33.071981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:33.176932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8a29a-1be8-4f2a-8838-2d574467a12a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:33.215163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:33.215815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:33.216097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:33.216374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:33.217128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:33.217445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:33.218850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:33.220289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:33.228436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 583/1157] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:33.235105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:33.330990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:33.339293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f058be-74b4-4e98-80d3-41833a348190 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:33.377036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:33.377935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:33.378212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:33.378484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] PATH_INFO: `/v3/users/a5b3ca9b45d54d1ca88b1d1313ca3d03` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:33.379703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] RBAC: Authorizing `identity:get_user(user_id=a5b3ca9b45d54d1ca88b1d1313ca3d03)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:33.381126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:33.382784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 575/1158] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:33 2022] GET /identity/v3/users/a5b3ca9b45d54d1ca88b1d1313ca3d03 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:33.389347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:33.499344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04296e7f-c390-47ad-bc67-0cba0ece7975 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:33.536466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:33.537060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:33.537331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:33.537643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:33.606621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:33.616168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:33.617676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:33.724689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 584/1159] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:19:33.732822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:33.846406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee7ef-883d-4947-974a-77c783112f2b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:33.884463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086b9ad-0e03-4ba0-be2a-fa3c601fb619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:33.885322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086b9ad-0e03-4ba0-be2a-fa3c601fb619 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:33.885633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086b9ad-0e03-4ba0-be2a-fa3c601fb619 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:33.885904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086b9ad-0e03-4ba0-be2a-fa3c601fb619 None admin] PATH_INFO: `/v3/projects/be6fab1fef114a6b8040bc07420d8587` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:33.888195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086b9ad-0e03-4ba0-be2a-fa3c601fb619 None admin] RBAC: Authorizing `identity:get_project(project_id=be6fab1fef114a6b8040bc07420d8587)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:33.890247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086b9ad-0e03-4ba0-be2a-fa3c601fb619 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:33.891956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 576/1160] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:33 2022] GET /identity/v3/projects/be6fab1fef114a6b8040bc07420d8587 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:33.898354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:34.008926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:34.017674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841ad74-3c5c-4934-82e3-d6856690a51a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:34.058718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:34.059371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:34.059652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:34.059926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:34.062049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=be6fab1fef114a6b8040bc07420d8587, user.id=a5b3ca9b45d54d1ca88b1d1313ca3d03) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:34.062363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:34.063869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e784dc-7613-488f-b8c8-05d8281fbe10 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:34.070338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 585/1161] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:33 2022] GET /identity/v3/role_assignments?user.id=a5b3ca9b45d54d1ca88b1d1313ca3d03&scope.project.id=be6fab1fef114a6b8040bc07420d8587&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:36.349149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959878f1-b61d-4e12-b2d6-627585831290 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:36.349745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959878f1-b61d-4e12-b2d6-627585831290 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:36.349745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959878f1-b61d-4e12-b2d6-627585831290 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:36.351124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 577/1162] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:36.358067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:36.358829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:36.358997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:36.387207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:36.388397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:36.461357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 586/1163] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:36.490980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959878f1-b61d-4e12-b2d6-627585831290 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:36.524837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:36.603227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959878f1-b61d-4e12-b2d6-627585831290 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:36.637835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:36.638456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:36.638738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:36.639011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:36.707654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:36.709094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:36.738242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:36.813669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 578/1164] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:19:36.816640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:36.908729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:36.920512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a551a-3b78-4100-8976-cb91d8082d67 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:36.955475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:36.956282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:36.956564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:36.956843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:36.963708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:36.965206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3a8799c2-f2a2-4b84-b0fb-d0ecc235e6e8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:36.979302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:36.980712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:36.981820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 587/1165] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:19:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:36.986826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:37.093190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4a0ff-a5b2-4193-b6b9-ace6dc953e97 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:37.122157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:37.128844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:37.129527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:37.129793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:37.130069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:37.130790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:37.131093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:37.132449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81c44a-34a5-4920-89a7-16344fe53e0b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:37.140202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 579/1166] 10.222.0.24 () {56 vars in 1115 bytes} [Thu May 19 13:19:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:39.329285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944a1f-e060-447e-93ab-8d0bce46fa2a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:39.330154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944a1f-e060-447e-93ab-8d0bce46fa2a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:39.330329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad944a1f-e060-447e-93ab-8d0bce46fa2a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:39.332435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 588/1167] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:39.338885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7e98a-05dc-4423-b9a0-95f144b4eccc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:39.339634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7e98a-05dc-4423-b9a0-95f144b4eccc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:39.339783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7e98a-05dc-4423-b9a0-95f144b4eccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:39.365168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-55f7e98a-05dc-4423-b9a0-95f144b4eccc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:39.366389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-55f7e98a-05dc-4423-b9a0-95f144b4eccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:39.442023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 580/1168] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:39.465186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944a1f-e060-447e-93ab-8d0bce46fa2a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:39.584499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad944a1f-e060-447e-93ab-8d0bce46fa2a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:39.619862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e525c0f-d78f-4698-a23f-177e74d953fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:39.620504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e525c0f-d78f-4698-a23f-177e74d953fa None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:39.620980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e525c0f-d78f-4698-a23f-177e74d953fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:39.621139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e525c0f-d78f-4698-a23f-177e74d953fa None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:39.621940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e525c0f-d78f-4698-a23f-177e74d953fa None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:39.623311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e525c0f-d78f-4698-a23f-177e74d953fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:39.654164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 589/1169] 10.222.0.24 () {60 vars in 1138 bytes} [Thu May 19 13:19:39 2022] POST /identity/v3/domains => generated 267 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:19:41.856924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813c130-b461-43b0-94b1-5910f5d55457 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:41.857841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813c130-b461-43b0-94b1-5910f5d55457 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:41.857841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813c130-b461-43b0-94b1-5910f5d55457 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:41.860076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 581/1170] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:41.866442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:41.867158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:41.867328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:41.886729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:41.887904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:41.890837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:41.960458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 590/1171] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:41.984784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813c130-b461-43b0-94b1-5910f5d55457 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:42.100211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813c130-b461-43b0-94b1-5910f5d55457 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:42.124885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:42.141114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025610-fc68-44ec-8bd0-1c9f1553f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:42.142540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025610-fc68-44ec-8bd0-1c9f1553f276 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:42.143005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025610-fc68-44ec-8bd0-1c9f1553f276 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:42.143456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025610-fc68-44ec-8bd0-1c9f1553f276 None admin] PATH_INFO: `/v3/domains/c6858bb4b4f14f4b809dbf4e62f997ff` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:42.159261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025610-fc68-44ec-8bd0-1c9f1553f276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c6858bb4b4f14f4b809dbf4e62f997ff)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:42.160710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025610-fc68-44ec-8bd0-1c9f1553f276 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:42.162408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 582/1172] 10.222.0.24 () {56 vars in 1183 bytes} [Thu May 19 13:19:41 2022] GET /identity/v3/domains/c6858bb4b4f14f4b809dbf4e62f997ff => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:42.169423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:42.273986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e4058-fd9c-4e1c-b860-a2cea878aa86 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:42.309629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420a260-857e-49d8-92a5-1bdd25a16a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:42.310274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420a260-857e-49d8-92a5-1bdd25a16a79 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:42.310554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420a260-857e-49d8-92a5-1bdd25a16a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:42.310822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420a260-857e-49d8-92a5-1bdd25a16a79 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:42.311526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420a260-857e-49d8-92a5-1bdd25a16a79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:42.312962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420a260-857e-49d8-92a5-1bdd25a16a79 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:42.323127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:42.340748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 591/1173] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:19:42 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:19:44.578353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c514-3e75-4498-9590-24d9998a60b2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:44.578893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c514-3e75-4498-9590-24d9998a60b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:44.578893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3c514-3e75-4498-9590-24d9998a60b2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:44.580178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 583/1174] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:44.586578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:44.587033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:44.587162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:44.613191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:44.614458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:44.695766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 592/1175] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:44.721064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c514-3e75-4498-9590-24d9998a60b2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:44.834470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3c514-3e75-4498-9590-24d9998a60b2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:44.878629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031babe-02df-4a38-bc99-806b5f64bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:44.879927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031babe-02df-4a38-bc99-806b5f64bfcf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:44.880362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031babe-02df-4a38-bc99-806b5f64bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:44.880788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031babe-02df-4a38-bc99-806b5f64bfcf None admin] PATH_INFO: `/v3/domains/c6858bb4b4f14f4b809dbf4e62f997ff` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:44.882983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031babe-02df-4a38-bc99-806b5f64bfcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=c6858bb4b4f14f4b809dbf4e62f997ff)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:44.885312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031babe-02df-4a38-bc99-806b5f64bfcf None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:44.888039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 584/1176] 10.222.0.24 () {56 vars in 1183 bytes} [Thu May 19 13:19:44 2022] GET /identity/v3/domains/c6858bb4b4f14f4b809dbf4e62f997ff => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:44.894738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:45.011605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd10360-521d-4fd0-af65-922c41987f6c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:45.037189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:45.051005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7b5b-e42d-47ce-b074-8a6392473368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:45.051674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7b5b-e42d-47ce-b074-8a6392473368 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:45.051962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7b5b-e42d-47ce-b074-8a6392473368 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:45.052240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7b5b-e42d-47ce-b074-8a6392473368 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:45.052951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7b5b-e42d-47ce-b074-8a6392473368 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:45.054437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7b5b-e42d-47ce-b074-8a6392473368 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:45.075994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 593/1177] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:19:44 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:19:47.348014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835dcd8-8b96-456e-b760-8776e2302857 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:47.348875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835dcd8-8b96-456e-b760-8776e2302857 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:47.348875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835dcd8-8b96-456e-b760-8776e2302857 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:47.351148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 585/1178] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:47.357597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:47.358321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:47.358515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:47.380700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:47.381908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:47.455769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 594/1179] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:47.489020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835dcd8-8b96-456e-b760-8776e2302857 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:47.549835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:47.609175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835dcd8-8b96-456e-b760-8776e2302857 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:47.644247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:47.645031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:47.645305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:47.645607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:47.656657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:47.664022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:47.665441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:47.666465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 586/1180] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:47.672476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:47.775655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f1294-5b3e-49b9-87e3-cd4c729cc1c3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:47.785161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:47.812583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:47.813187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:47.813491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:47.813776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:47.814540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:47.814848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:47.816231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:47.829034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 595/1181] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:47.835130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:47.936348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a121-201a-4a4d-b683-7b93b403b2cf None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:47.950554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:47.977589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:47.978442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:47.978718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:47.978979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] PATH_INFO: `/v3/users/4f3108bf66f54521927b3a3c10e5921f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:47.991995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] RBAC: Authorizing `identity:get_user(user_id=4f3108bf66f54521927b3a3c10e5921f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:47.993449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:47.995096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 587/1182] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:47 2022] GET /identity/v3/users/4f3108bf66f54521927b3a3c10e5921f => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:48.004529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:48.123501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ab2da-6d28-4149-af62-df9e47c184b9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:48.160021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:48.160664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:48.160946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:48.161226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:48.203429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:48.237557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:48.239035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:48.344189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 596/1183] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:19:48.350491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:48.456544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ebec1-aed2-402d-965c-07cd0d16583b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:48.490824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b7a21-aeb2-414b-b688-c160f7707b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:48.491662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b7a21-aeb2-414b-b688-c160f7707b0d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:48.491936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b7a21-aeb2-414b-b688-c160f7707b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:48.492200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b7a21-aeb2-414b-b688-c160f7707b0d None admin] PATH_INFO: `/v3/projects/85e782d1f8b843b79913237f21a36e65` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:48.493442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b7a21-aeb2-414b-b688-c160f7707b0d None admin] RBAC: Authorizing `identity:get_project(project_id=85e782d1f8b843b79913237f21a36e65)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:48.494852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b7a21-aeb2-414b-b688-c160f7707b0d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:48.496646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 588/1184] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:48 2022] GET /identity/v3/projects/85e782d1f8b843b79913237f21a36e65 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:48.502800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:48.575514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:48.613173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0c53-0090-4dab-833b-0e8c5033432d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:48.648877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:48.649548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:48.649830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:48.650102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:48.650938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=85e782d1f8b843b79913237f21a36e65, user.id=4f3108bf66f54521927b3a3c10e5921f) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:48.651249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:48.652783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e45256-3c07-4c4f-a21e-321e9b87cc60 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:48.658391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 597/1185] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:48 2022] GET /identity/v3/role_assignments?user.id=4f3108bf66f54521927b3a3c10e5921f&scope.project.id=85e782d1f8b843b79913237f21a36e65&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:50.808439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b62cff-1c49-4036-a581-711374f799d4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:50.809318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b62cff-1c49-4036-a581-711374f799d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:50.809318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b62cff-1c49-4036-a581-711374f799d4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:50.811564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 589/1186] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:50.817885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:50.818576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:50.818761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:50.844352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:50.845596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:50.918912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 598/1187] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:19:50.946042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b62cff-1c49-4036-a581-711374f799d4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:51.001310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:51.064973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b62cff-1c49-4036-a581-711374f799d4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:51.101786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:51.102566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:51.102846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:51.103120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:51.114413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:51.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:51.123162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:19:51.124660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 590/1188] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:51.132642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:51.208400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:51.260186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aeb0e-3829-45a1-bb2b-6c9f536e7d6b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:51.294519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:51.295173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:51.295487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:51.295762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:51.296516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:51.296821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:51.298230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:51.306111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 599/1189] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:51.312462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:51.411006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:51.415176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b75b60-33a3-469f-b707-bc5b3a45ebad None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:51.453558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:51.454418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:51.454690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:51.454961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] PATH_INFO: `/v3/users/4f3108bf66f54521927b3a3c10e5921f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:51.456211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] RBAC: Authorizing `identity:get_user(user_id=4f3108bf66f54521927b3a3c10e5921f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:51.457669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:51.459265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 591/1190] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:51 2022] GET /identity/v3/users/4f3108bf66f54521927b3a3c10e5921f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:51.465653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:51.572535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b281-9d89-4cd1-9d1d-57ad19360e0a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:51.598498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:51.611887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787971-55e6-492c-a94e-27b310fe6fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:51.613189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787971-55e6-492c-a94e-27b310fe6fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:51.613607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787971-55e6-492c-a94e-27b310fe6fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:51.614001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787971-55e6-492c-a94e-27b310fe6fa1 None admin] PATH_INFO: `/v3/projects/85e782d1f8b843b79913237f21a36e65` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:51.615983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787971-55e6-492c-a94e-27b310fe6fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=85e782d1f8b843b79913237f21a36e65)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:51.618358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787971-55e6-492c-a94e-27b310fe6fa1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:51.621060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 600/1191] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:51 2022] GET /identity/v3/projects/85e782d1f8b843b79913237f21a36e65 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:51.627152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:51.731198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cb23c-dc4c-4320-92a8-e0823272598b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:51.768707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73358cb-d65e-4999-9121-56ae2ef69bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:51.776976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73358cb-d65e-4999-9121-56ae2ef69bef None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:51.777254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73358cb-d65e-4999-9121-56ae2ef69bef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:51.777597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73358cb-d65e-4999-9121-56ae2ef69bef None admin] PATH_INFO: `/v3/projects/85e782d1f8b843b79913237f21a36e65/users/4f3108bf66f54521927b3a3c10e5921f/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:51.779740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73358cb-d65e-4999-9121-56ae2ef69bef None admin] RBAC: Authorizing `identity:create_grant(project_id=85e782d1f8b843b79913237f21a36e65, user_id=4f3108bf66f54521927b3a3c10e5921f, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:51.781169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73358cb-d65e-4999-9121-56ae2ef69bef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:51.804051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 592/1192] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:19:51 2022] PUT /identity/v3/projects/85e782d1f8b843b79913237f21a36e65/users/4f3108bf66f54521927b3a3c10e5921f/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:19:53.967596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfd1f0-b429-474e-a79b-d9e64a6cea82 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:53.968235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfd1f0-b429-474e-a79b-d9e64a6cea82 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:53.968235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfd1f0-b429-474e-a79b-d9e64a6cea82 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:53.969499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 601/1193] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:19:53.977612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:53.978045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:53.978144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:53.996912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:19:53.998156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:19:54.019435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:54.068693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 593/1194] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:19:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:19:54.092034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfd1f0-b429-474e-a79b-d9e64a6cea82 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:54.203393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfd1f0-b429-474e-a79b-d9e64a6cea82 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:54.250102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:54.250890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:54.251171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:54.251445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:54.263015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:19:54.270561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:19:54.271653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:19:54.272523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 602/1195] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:19:54.279674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:54.382627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3567a2-4064-4a2e-8011-f3975c11714b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:54.418523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:54.419361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:54.419645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:54.419915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:54.420669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:54.420974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:54.422378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:54.430609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 594/1196] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:54.440440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:54.501936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:54.547915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2d269-6c97-468a-8614-d5e6a5650592 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:54.583365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:54.584214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:54.584496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:54.584757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] PATH_INFO: `/v3/users/4f3108bf66f54521927b3a3c10e5921f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:54.586108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] RBAC: Authorizing `identity:get_user(user_id=4f3108bf66f54521927b3a3c10e5921f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:54.587557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:54.589169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 603/1197] 10.222.0.24 () {56 vars in 1177 bytes} [Thu May 19 13:19:54 2022] GET /identity/v3/users/4f3108bf66f54521927b3a3c10e5921f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:54.595871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:54.616337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:54.704237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bafb1c-b1b3-4f00-b2f4-16d7b70c6d2c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:54.739888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:54.740524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:54.740813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:54.741085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:54.828089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:54.829550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:54.850097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:54.945172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 595/1198] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:19:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 13:19:54.951558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:55.061018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0207eab-2376-4961-a189-a87c2df4203e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:55.083667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:55.096920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52491992-8aaf-4b36-b2f9-ca85f4373dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:55.097827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491992-8aaf-4b36-b2f9-ca85f4373dfa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:55.098112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491992-8aaf-4b36-b2f9-ca85f4373dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:55.098383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52491992-8aaf-4b36-b2f9-ca85f4373dfa None admin] PATH_INFO: `/v3/projects/85e782d1f8b843b79913237f21a36e65` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:55.099582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52491992-8aaf-4b36-b2f9-ca85f4373dfa None admin] RBAC: Authorizing `identity:get_project(project_id=85e782d1f8b843b79913237f21a36e65)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:55.101038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52491992-8aaf-4b36-b2f9-ca85f4373dfa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:55.102754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 604/1199] 10.222.0.24 () {56 vars in 1186 bytes} [Thu May 19 13:19:54 2022] GET /identity/v3/projects/85e782d1f8b843b79913237f21a36e65 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:19:55.109320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:19:55.207733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02fdd-6293-48a7-8415-20f69d94ffa5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:19:55.226893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:19:55.248295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:19:55.248945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:19:55.249224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:19:55.249504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:19:55.250341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=85e782d1f8b843b79913237f21a36e65, user.id=4f3108bf66f54521927b3a3c10e5921f) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:19:55.250650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:19:55.252075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ebbd6-a815-498a-ae1e-e77efa3e7335 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:19:55.260079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 596/1200] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:19:55 2022] GET /identity/v3/role_assignments?user.id=4f3108bf66f54521927b3a3c10e5921f&scope.project.id=85e782d1f8b843b79913237f21a36e65&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:20:05.092307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:20:05.255450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=9 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:20:56.286555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce276-a593-4bf9-a659-69bb2892a9af None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:56.287164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce276-a593-4bf9-a659-69bb2892a9af None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:56.287164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce276-a593-4bf9-a659-69bb2892a9af None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:56.288460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 605/1201] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:20:56.294648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:56.295321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:56.295321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:56.318340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:20:56.319558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:20:56.403433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 597/1202] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:20:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:20:56.428464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbce276-a593-4bf9-a659-69bb2892a9af None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:20:56.569760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbce276-a593-4bf9-a659-69bb2892a9af None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:20:56.612635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:20:56.613466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:56.613756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:56.614032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:56.630010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:20:56.631668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:20:56.651399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:20:56.658628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:20:56.659897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:20:56.659897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:20:56.659897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:20:56.659897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:20:56.659897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 606/1203] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:20:56.664842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:20:56.783459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b31eb-0537-436a-ada5-7e03f852adc1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:20:56.819030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:20:56.825755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:20:56.826287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:56.826866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:56.827311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:56.828104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:20:56.829222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:20:56.830146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8761f92-ada6-4291-b10e-4306a00a07ce None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:20:56.848434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 598/1204] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:20:56.855677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:20:56.919317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:20:56.977986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26abef-d753-4819-865c-09ab86af78fe None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:20:57.020467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c9b44-2d36-495f-b67d-7097b6b536e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:20:57.021184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c9b44-2d36-495f-b67d-7097b6b536e7 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:57.021502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c9b44-2d36-495f-b67d-7097b6b536e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:57.021814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c9b44-2d36-495f-b67d-7097b6b536e7 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:57.022549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c9b44-2d36-495f-b67d-7097b6b536e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:20:57.024014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c9b44-2d36-495f-b67d-7097b6b536e7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:20:57.045151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 607/1205] 10.222.0.24 () {60 vars in 1132 bytes} [Thu May 19 13:20:56 2022] POST /identity/v3/users => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:20:59.355670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578711fa-2f8a-4c9c-8c00-a3fdb8f1d48d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:59.356267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578711fa-2f8a-4c9c-8c00-a3fdb8f1d48d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:59.356267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578711fa-2f8a-4c9c-8c00-a3fdb8f1d48d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:59.357906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 599/1206] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:20:59.363722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:59.364170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:59.364293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:59.382174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:20:59.383643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:20:59.463679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 608/1207] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:20:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:20:59.488857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578711fa-2f8a-4c9c-8c00-a3fdb8f1d48d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:20:59.617430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578711fa-2f8a-4c9c-8c00-a3fdb8f1d48d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:20:59.667658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:20:59.668547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:59.668832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:59.669107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:59.680577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:20:59.688632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:20:59.689999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:20:59.690918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 600/1208] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:20:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:20:59.697900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:20:59.744738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:20:59.824907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b36d7-9407-4cea-a2fd-d09d725e4c53 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:20:59.866544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:20:59.867784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:20:59.867784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:20:59.867784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:20:59.868451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:20:59.868756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:20:59.870324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:20:59.881857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 609/1209] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:20:59.887652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:20:59.944812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:00.007446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edbd27-f960-4dca-b307-9ee95636705b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:00.050431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:00.051117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:00.051473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:00.051759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:00.064721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:00.066244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:00.085876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:00.086907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:00.087851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:00.087851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:00.087851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:00.087851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:00.087851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 601/1210] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:00.093773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:00.215827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:00.218964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809b030-bdb6-450e-bdff-9c79175b13e0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:00.264827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:00.265531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:00.265812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:00.267551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:00.267551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:00.267551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:00.268465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:00.287532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 610/1211] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:00.289503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:00.399240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa2a21-5527-43ea-b999-2298c101028d None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:00.400992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:00.438233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:00.439067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:00.439357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:00.439669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:00.448585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:00.450116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:21:00.467912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:00.469119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:00.470201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 602/1212] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:00.475524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:00.593666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf12fc-b979-4b55-8568-84dfe69d1bd4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:00.628269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:00.628935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:00.629215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:00.629512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:00.630350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:00.630651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:00.632091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:00.655899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 611/1213] 10.222.0.24 () {56 vars in 1143 bytes} [Thu May 19 13:21:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:00.657717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:00.767077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b81ca-d54b-4c38-9207-b3207991d993 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:00.805074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:00.805761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:00.806131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:00.806290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:00.841222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:00.884459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:00.886267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:01.001107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 603/1214] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:21:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:21:01.007350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:01.033634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:01.124794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4c35c-9ff9-4a80-97d9-a9d2573587c3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:01.162247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:01.163135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:01.163325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:01.163600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:01.175831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:01.177318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:01.196596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:01.197801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:01.198963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:01.198963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:01.198963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:01.198963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:01.198963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 612/1215] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:01.206363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:01.284787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:01.331463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e10e1-014d-43d4-b552-cacc1706bf85 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:01.371970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:01.373060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:01.373346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:01.373658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:01.374390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:01.374701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:01.376074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:01.388807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 604/1216] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:01.396388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:01.499358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:01.535971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602b7f-8ec9-43ad-b099-0a3cff4aee3e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:01.574964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:01.575944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:01.576328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:01.576754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:01.584765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:01.586466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:21:01.601824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:01.603020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:01.604207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:01.610367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 613/1217] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:21:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:21:01.611724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:01.721756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750842fe-c2f3-493b-8294-c0a7a1a83ff8 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:01.751308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:01.765323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:01.766183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:01.766466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:01.766740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:01.767540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:01.767854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:01.769297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d982c8-8457-4fcb-9c8d-115d356ec6e5 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:01.782783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 605/1218] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:01.789508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:01.900751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8759-5eff-4bb5-a29b-7bc5e0d70fbc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:01.932621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:01.939763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:01.940645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:01.941152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:01.941506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:01.942375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=48ab1e052fff48bfbbc222528424b79c) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:01.942736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:01.944266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c72e63-ea6a-4714-aef2-8979ec03e232 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:01.950690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 614/1219] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:21:01 2022] GET /identity/v3/role_assignments?user.id=48ab1e052fff48bfbbc222528424b79c&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:04.174417 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d05d77-28f0-41fb-9dd8-019924b848bf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:04.174946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d05d77-28f0-41fb-9dd8-019924b848bf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:04.174946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d05d77-28f0-41fb-9dd8-019924b848bf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:04.176324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 606/1220] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:04.183142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:04.183636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:04.183736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:04.200683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:04.201907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:04.275106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 615/1221] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:04.297893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d05d77-28f0-41fb-9dd8-019924b848bf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:04.404031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d05d77-28f0-41fb-9dd8-019924b848bf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:04.437365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:04.438107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:04.438382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:04.438647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:04.448442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:21:04.455880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:04.457055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:04.458237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 607/1222] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:21:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:04.463357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:04.570124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a79d5-baef-4759-bfde-891969507c18 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:04.605898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:04.617844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:04.618519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:04.618812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:04.619099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:04.620072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:04.620731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:04.622737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:04.634143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 616/1223] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:21:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:04.641359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:04.688054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:04.757535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717e11-8b98-4034-a47b-7794661c1ee5 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:04.796520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:04.797734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:04.798217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:04.798661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:04.811592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:04.813033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:04.832588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:04.833860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:04.835006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:04.835006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:04.835006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:04.835006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:04.835006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 608/1224] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:04.839839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:04.946040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce14aa-f00f-4556-b817-7dbf51609777 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:04.980103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:04.980799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:04.981086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:04.981365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:04.982142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:04.982453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:04.983856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:04.996415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 617/1225] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:05.002155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:05.111068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:05.119624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105263e4-e5a4-48d7-9396-774769b523a1 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:05.154888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:05.155703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:05.155967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:05.156237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:05.163962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:05.165458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:21:05.180395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:05.182094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:05.183477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:05.186569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 609/1226] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:05.187683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:05.236906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:05.292668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c99e7-2b5b-4f9d-bd20-5f1016fabe29 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:05.333540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:05.334217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:05.334508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:05.334773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:05.335605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:05.335908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:05.337355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:05.357364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 618/1227] 10.222.0.24 () {56 vars in 1143 bytes} [Thu May 19 13:21:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:05.363048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:05.472337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16885cc2-f1a4-443b-ac9b-bc3b73b3588c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:05.511700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:05.512079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:05.512366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:05.512638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:05.524521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:05.526326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:05.547129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:05.548406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:05.549593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:05.549593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:05.549593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:05.549593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:05.549593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 610/1228] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:05.554659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:05.663641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4da05-d743-4801-b418-de632c915371 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:05.666293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:05.698081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:05.698790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:05.699072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:05.699341 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:05.700055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:05.700357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:05.701748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:05.713184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 619/1229] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:05.719375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:05.729266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:05.827806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f1161-728f-46f0-85c6-3f770463e20e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:05.861909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:05.862695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:05.862973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:05.863242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:05.870567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:05.872077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:21:05.886619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:05.887736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:05.888799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 611/1230] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:05.894054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:06.003217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d65f6-f482-4ada-b92b-2c34cf667489 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:06.042078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:06.042718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:06.043007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:06.043275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:06.044049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:06.044358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:06.045820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c538ee-69d2-4125-83ac-49bb853089a0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:06.062815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 620/1231] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:06.068975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:06.157111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:06.194927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33b3c8-030e-49aa-b6c6-5dfc48ae57df None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:06.231364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d7bf-29d0-43ea-b2d6-eda7cd2a6b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:06.232263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d7bf-29d0-43ea-b2d6-eda7cd2a6b62 None admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:06.232550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d7bf-29d0-43ea-b2d6-eda7cd2a6b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:06.232814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d7bf-29d0-43ea-b2d6-eda7cd2a6b62 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/48ab1e052fff48bfbbc222528424b79c/roles/d1dcb7c1298146f59d801c5e3f8b58f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:06.248285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d7bf-29d0-43ea-b2d6-eda7cd2a6b62 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=48ab1e052fff48bfbbc222528424b79c, role_id=d1dcb7c1298146f59d801c5e3f8b58f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:06.249807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d7bf-29d0-43ea-b2d6-eda7cd2a6b62 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:06.266501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 612/1232] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:21:06 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/48ab1e052fff48bfbbc222528424b79c/roles/d1dcb7c1298146f59d801c5e3f8b58f2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:21:08.643721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf26933-ec3b-49d3-bf37-adb11b151a4f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:08.644294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf26933-ec3b-49d3-bf37-adb11b151a4f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:08.644294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf26933-ec3b-49d3-bf37-adb11b151a4f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:08.645812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 621/1233] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:08.651933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:08.652463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:08.652720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:08.672634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:08.674264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:08.760628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 613/1234] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:08.789143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf26933-ec3b-49d3-bf37-adb11b151a4f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:08.819890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:08.909851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf26933-ec3b-49d3-bf37-adb11b151a4f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:08.947618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:08.948463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:08.948937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:08.949353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:08.960581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:21:08.968379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:08.969540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:08.975395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 622/1235] 10.222.0.24 () {56 vars in 1102 bytes} [Thu May 19 13:21:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:08.976331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:09.029738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:09.091147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29374305-a9ee-431f-b069-0b22e8d23245 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:09.125236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:09.125981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:09.126265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:09.126604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:09.127476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:09.127953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:09.129468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:09.139501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 614/1236] 10.222.0.24 () {56 vars in 1103 bytes} [Thu May 19 13:21:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:09.145730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:09.233289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:09.251765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad68449-8a2a-4056-9c85-53775301dab2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:09.288609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:09.289431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:09.289710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:09.289973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:09.300958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:09.302439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:09.321183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:09.322384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:09.323884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:09.323884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:09.323884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:09.323884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:09.323884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 623/1237] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:09.328381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:09.437542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90d892-1ba5-4596-8917-07f37d2e80f3 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:09.447017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:09.477373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:09.478245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:09.478472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:09.478723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:09.479559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:09.479834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:09.481320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:09.492982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 615/1238] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:09.498872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:09.613860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193f71a-1099-4c98-af1b-f1010a0a1d25 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:09.652221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:09.653030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:09.653313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:09.653624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:09.656995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:09.662810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:09.664480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:21:09.679631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:09.680815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:09.681825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 624/1239] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:09.686850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:09.789306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa7aeb-229c-4c6e-b073-612b388d5602 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:09.825209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:09.825919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:09.826202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:09.826482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:09.827329 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:09.827638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:09.829067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:09.847833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 616/1240] 10.222.0.24 () {56 vars in 1143 bytes} [Thu May 19 13:21:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:09.853574 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:09.965117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9f93-0e21-4014-ae03-e731c4eb4157 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:10.000007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:10.000652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:10.000937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:10.001242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:10.071787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:10.081313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:10.082808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:10.193226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 625/1241] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:21:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:21:10.198963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:10.228316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:10.304643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f68077-dc03-4158-9b94-db873d975e12 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:10.342870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:10.343736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:10.346752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:10.346752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:10.356772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:10.358261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:10.376801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:10.382064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:10.382064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:10.382064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:10.382064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:10.382814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 617/1242] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:10.383889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:10.488132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:10.496099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4c402-4f3c-40d7-81c0-161562fc6cc5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:10.540522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:10.541582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:10.541889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:10.542146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:10.542880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:10.543186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:10.545013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:10.559668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 626/1243] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:10.565474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:10.651353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:10.676724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4be78c-24e3-4e43-9707-6620f4535b2b None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:10.714095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:10.715022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:10.715208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:10.715480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:10.722305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:10.724086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:21:10.740754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:10.741787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:10.742933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:10.743488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 618/1244] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:21:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:10.750600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:10.856090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ba88d-31e1-4f05-a9cf-4c7d5cb29b5a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:10.892126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:10.892783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:10.893065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:10.893336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:10.896120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:10.896428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:10.897988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51b23c-87ff-4268-818b-0b502b3fb7ea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:10.911602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 627/1245] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:21:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:10.918258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:11.026075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26cdbd-e40b-4d27-a225-cbfb3815b007 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:11.068203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:11.069662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:11.070109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:11.070528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:11.071857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] RBAC: Adding query filter params (role.id=d1dcb7c1298146f59d801c5e3f8b58f2, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=48ab1e052fff48bfbbc222528424b79c) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:11.072262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:11.074803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148d0ac-842b-41ba-9807-9fbc90aabe41 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:11.084234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 619/1246] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:21:10 2022] GET /identity/v3/role_assignments?user.id=48ab1e052fff48bfbbc222528424b79c&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=d1dcb7c1298146f59d801c5e3f8b58f2 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:13.346305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2669af-15c5-4355-bba9-d35340ef929d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:13.346843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2669af-15c5-4355-bba9-d35340ef929d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:13.346843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2669af-15c5-4355-bba9-d35340ef929d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:13.348415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 628/1247] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:13.353636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:13.354115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:13.354218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:13.360191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:13.374783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:13.376149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:13.456140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 620/1248] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:13.479666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2669af-15c5-4355-bba9-d35340ef929d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:13.501241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:13.599178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2669af-15c5-4355-bba9-d35340ef929d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:13.638097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:13.638898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:13.640279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:13.640559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:13.653751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:21:13.661328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:13.662405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:13.663319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 629/1249] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:13.669262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:13.782382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfac7b-8787-4269-9ffd-c2388a46aa42 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:13.816297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:13.822332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:13.823357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:13.823641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:13.823917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:13.824671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:13.824976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:13.826388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:13.835188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 621/1250] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:13.841834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:13.932028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:13.959920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024b3fa-a39a-470f-bbcb-688fb79ff23c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:14.001218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:14.002150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:14.002397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:14.002670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:14.014006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:14.015467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:14.040079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:14.041559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:14.042606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:14.042606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:14.042606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:14.042606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:14.042606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 630/1251] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:14.049051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:14.176297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0248d9-fb45-4175-be54-72ab9893b344 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:14.212811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:14.213492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:14.213998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:14.214355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:14.215095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:14.215442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:14.216816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:14.230348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 622/1252] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:14.236040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:14.358001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd0d76-f5c3-4fc1-b613-2d5509467b8b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:14.387198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:14.398204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:14.399365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:14.399799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:14.400255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:14.408943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:14.410788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:21:14.429454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:14.430867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:14.431985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 631/1253] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:14.437847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:14.499340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:14.556056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae04d49-219d-413e-8d05-5b31889c1b14 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:14.596985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:14.597707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:14.597990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:14.598315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:14.599353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:14.599671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:14.601103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:14.621883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 623/1254] 10.222.0.24 () {56 vars in 1143 bytes} [Thu May 19 13:21:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:14.627441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:14.750446 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75e972-746d-494a-9cb1-f86af17ce78b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:14.789879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:14.790525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:14.791136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:14.791415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:14.854580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:14.868924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:14.871325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:14.975626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 632/1255] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:21:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:21:14.981360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:15.099311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bbcc0-5311-4c56-a72b-dc607372404e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:15.112836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:15.134635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:15.135410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:15.135731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:15.136024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:15.146216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:15.147679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:15.166638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:15.168215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:15.169259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:15.169259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:15.169259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:15.169259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:15.169259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 624/1256] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:15.174476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:15.272799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:15.287097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfef444-446f-4d41-9f75-6bb07daf98dd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:15.325862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:15.326527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:15.326804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:15.327077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:15.327801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:15.328093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:15.329576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:15.343236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 633/1257] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:15.349450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:15.470139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5876cb-c350-4b5a-b6d6-35ddc9ce0636 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:15.508166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:15.508984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:15.509261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:15.509568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:15.517909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:15.519388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:21:15.534487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:15.535635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:15.536769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 625/1258] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:21:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:15.544128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:15.656488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b33c7-caa7-4646-aef2-b55477907c0c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:15.695536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:15.696207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:15.696490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:15.696765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:15.697593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:15.697906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:15.699455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cebaf-c5d7-4301-a9ea-ea23ce694be5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:15.717907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 634/1259] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:21:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:15.719221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:15.748942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:15.829894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4cae4-bb46-40be-a35f-bc9ca6750011 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:15.866626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:15.867291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:15.867574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:15.867812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:15.868688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=48ab1e052fff48bfbbc222528424b79c) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:15.868999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:15.870781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c991d-b84a-4e24-bdb4-ee2073332b3e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:15.877362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 626/1260] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:21:15 2022] GET /identity/v3/role_assignments?user.id=48ab1e052fff48bfbbc222528424b79c&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:18.076787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135df1fd-e4c8-48ca-b4ba-6e27d5745a59 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:18.077476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135df1fd-e4c8-48ca-b4ba-6e27d5745a59 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:18.077476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135df1fd-e4c8-48ca-b4ba-6e27d5745a59 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:18.078730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 635/1261] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:21:18.084863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:18.085275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:18.085450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:18.106905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:18.108095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:18.191664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 627/1262] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:21:18.217655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135df1fd-e4c8-48ca-b4ba-6e27d5745a59 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:18.229338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:18.333575 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135df1fd-e4c8-48ca-b4ba-6e27d5745a59 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:18.371408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:18.372196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:18.372477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:18.372748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:18.383388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:21:18.391332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:18.392431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:21:18.393587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:18.397592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 636/1263] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:21:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:18.399167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:18.426256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:18.520086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3cad01-5a58-4957-b327-cd837f93a0b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:18.556200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:18.556860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:18.557139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:18.557435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:18.558237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:18.558542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:18.559917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:18.568242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 628/1264] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:21:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:18.574359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:18.644423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:18.679688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf83c8-0a02-4d66-ae9b-d8d1849048e4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:18.717125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:18.717940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:18.718218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:18.718494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:18.729405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:18.731017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:18.748552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:18.749627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:18.750798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:18.750798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:18.750798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:18.750798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:18.750798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 637/1265] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:18.757554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:18.852456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:18.875062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2455e9-f261-4c19-abf7-b61166d23030 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:18.909348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:18.909961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:18.910224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:18.910501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:18.911230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:18.911536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:18.912908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:18.925027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 629/1266] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:18.934856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:19.054979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129a43c-346a-4786-a67d-b843631ccbfd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:19.077828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:19.095523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:19.096335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:19.096622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:19.096904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:19.105371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:19.106872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:21:19.121127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:19.122299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:19.123548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 638/1267] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:19.128282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:19.232622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fdeb2-f026-4c98-a3d7-039944da954e None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:19.269391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:19.270082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:19.270354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:19.270626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:19.271459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:19.271763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:19.273177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:19.282011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:19.291187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 630/1268] 10.222.0.24 () {56 vars in 1143 bytes} [Thu May 19 13:21:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:19.297192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:19.402741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49266df-7abf-4a9c-bb81-b41508850fc9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:19.446849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:19.447646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:19.447929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:19.448242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:19.459287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:19.460750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:19.478484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:19.479812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:19.481151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:19.481151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:19.481151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:19.481151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:19.484654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 639/1269] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:19.485682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:19.589857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37347437-b3f1-4299-840e-a128c2091079 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:19.626125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:19.626774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:19.627053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:19.627335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:19.628061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:19.628365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:19.629768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:19.643426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 631/1270] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:19.652435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:19.703246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:19.778999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50404c-58b2-440a-b549-0047c484e7d1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:19.814397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:19.815176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:19.815462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:19.815739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:19.822491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:19.824000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:21:19.838056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:19.839658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:19.840767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 640/1271] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:19.845236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:19.905346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:19.952648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f643502-3210-4a2a-a6ea-3b6db0874066 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:19.987259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:19.987909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:19.988193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:19.988467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:19.989255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:19.989593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:19.991029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b879-842f-40f4-8902-01505aa791fe None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:20.003007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 632/1272] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:20.009046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:20.113264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:20.124193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7b37a-7abc-4d6c-905d-e054671551e4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:20.161184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fd302-e6c3-4acc-a404-7cd6f92a2c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:20.162097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd302-e6c3-4acc-a404-7cd6f92a2c69 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:20.162377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd302-e6c3-4acc-a404-7cd6f92a2c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:20.162655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd302-e6c3-4acc-a404-7cd6f92a2c69 None admin] PATH_INFO: `/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/48ab1e052fff48bfbbc222528424b79c/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:20.164873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fd302-e6c3-4acc-a404-7cd6f92a2c69 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9852768855f4cc39dc219f0c0ece5c0, user_id=48ab1e052fff48bfbbc222528424b79c, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:20.166687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fd302-e6c3-4acc-a404-7cd6f92a2c69 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:20.182569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 641/1273] 10.222.0.24 () {58 vars in 1439 bytes} [Thu May 19 13:21:20 2022] PUT /identity/v3/projects/f9852768855f4cc39dc219f0c0ece5c0/users/48ab1e052fff48bfbbc222528424b79c/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:21:22.367373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e279a-e3bc-46ed-8e2c-7ce3470a4209 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:22.368199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e279a-e3bc-46ed-8e2c-7ce3470a4209 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:22.368199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e279a-e3bc-46ed-8e2c-7ce3470a4209 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:22.370439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 633/1274] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:22.376947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:22.377402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:22.377650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:22.397109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:22.398349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:22.469733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 642/1275] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:22.502950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e279a-e3bc-46ed-8e2c-7ce3470a4209 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:22.614257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e279a-e3bc-46ed-8e2c-7ce3470a4209 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:22.632216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:22.656167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:22.656956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:22.657243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:22.657556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:22.669110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 13:21:22.676677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:22.677900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:22.678869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 634/1276] 10.222.0.24 () {56 vars in 1096 bytes} [Thu May 19 13:21:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:22.684335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:22.769432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:22.790169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfa859-1738-4824-8410-f30ad505c0e9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:22.825514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:22.826164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:22.826440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:22.826713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:22.827448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:22.827753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:22.829122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:22.846071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 643/1277] 10.222.0.24 () {56 vars in 1099 bytes} [Thu May 19 13:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:22.852937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:22.954647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf357-7a49-4e90-8e55-a69a7e912ef2 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:22.989252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:22.990061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:22.990343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:22.990616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:23.001455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:23.002949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:23.020005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:23.021054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:23.022043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:23.022043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:23.022043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:23.022043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:23.022043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 635/1278] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:23.027188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:23.147405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0379e2-6b2c-4ee8-8b71-2510c4f754ae None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:23.184961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:23.185652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:23.185935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:23.186208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:23.186921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:23.187230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:23.188606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:23.200327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 644/1279] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:23.209370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:23.245619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:23.319298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e85e1c-59ca-4c37-8f8f-5e29949170a7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:23.353671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:23.354492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:23.354773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:23.355051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:23.362762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:23.364194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 13:21:23.378685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:23.380197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:23.382207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 636/1280] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:23.388939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:23.393031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:23.509382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53ed4d-1dac-45df-a5a2-86a40a58ebd2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:23.545622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:23.546287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:23.546567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:23.546840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:23.547683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:23.547989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:23.549452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:23.567046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 645/1281] 10.222.0.24 () {56 vars in 1143 bytes} [Thu May 19 13:21:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:23.575581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:23.671703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:23.686467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a6d42-4c2a-4153-9624-e45aca2abbdb None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:23.726716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:23.727361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:23.727643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:23.727917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:23.797680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:23.799063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:23.897571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:23.914810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 637/1282] 10.222.0.24 () {58 vars in 1282 bytes} [Thu May 19 13:21:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 13:21:23.921682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:23.994772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:24.037465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36507526-c8c9-43ea-bd78-f8a94d3833b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:24.082200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:24.082914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:24.083201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:24.083475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:24.094562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:24.096022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 13:21:24.114178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:24.115208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:24.116179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 13:21:24.116179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 13:21:24.116179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 13:21:24.116179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:24.116179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 646/1283] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:24.122773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:24.232543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3600e0-f34c-48ce-9e0d-d8bc5e2e3310 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:24.269079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:24.269773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:24.270058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:24.270333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:24.271062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:24.271370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:24.272752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:24.284609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 638/1284] 10.222.0.24 () {56 vars in 1109 bytes} [Thu May 19 13:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:24.290463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:24.408616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ed569-a939-4831-82b4-cb3219d07478 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:24.423053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:24.446731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:24.447501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:24.447778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:24.448043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:24.454934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:24.456386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 13:21:24.470024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:24.471051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application May 19 13:21:24.472022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 647/1285] 10.222.0.24 () {56 vars in 1111 bytes} [Thu May 19 13:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 13:21:24.477548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:24.532076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:24.590998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85b678-9b59-474b-947c-cbc9e59a7ae7 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:24.626577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:24.627227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:24.627502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:24.627766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:24.628550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:24.628856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:24.630307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f25603-9c04-4ad3-9b89-7860926067a8 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:24.644029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 639/1286] 10.222.0.24 () {56 vars in 1148 bytes} [Thu May 19 13:21:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:24.650362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:24.764637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566882a3-ce9d-4ff0-a7c0-22764ca79cf7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:24.799821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:24.800454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:24.800732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:24.801005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:24.801861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] RBAC: Adding query filter params (role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca, scope.project.id=f9852768855f4cc39dc219f0c0ece5c0, user.id=48ab1e052fff48bfbbc222528424b79c) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:24.802168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:24.803597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1028884-7f4e-4467-a3fc-cce3517fea7b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:24.809701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 648/1287] 10.222.0.24 () {56 vars in 1374 bytes} [Thu May 19 13:21:24 2022] GET /identity/v3/role_assignments?user.id=48ab1e052fff48bfbbc222528424b79c&scope.project.id=f9852768855f4cc39dc219f0c0ece5c0&role.id=0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:27.068796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9e186-65d1-410a-a97f-a09888de05d4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:27.069383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9e186-65d1-410a-a97f-a09888de05d4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:27.069383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9e186-65d1-410a-a97f-a09888de05d4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:27.071168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 640/1288] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:27.078555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:27.078967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:27.079257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:27.103754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:27.105610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:27.139372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:27.183339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 649/1289] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 13:21:27.211941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9e186-65d1-410a-a97f-a09888de05d4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:27.317720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9e186-65d1-410a-a97f-a09888de05d4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:27.328580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:27.355838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:27.356623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:27.356904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:27.357178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:27.357866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:27.359233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:27.371130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:27.372339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:27.373509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 641/1290] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:21:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:27.378526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:27.486964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a67c-9078-4ac7-90ef-187dc4840fb9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:27.522816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:27.523465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:27.523748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:27.524024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:27.524729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:27.525042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:27.526480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b1230-d9e9-484a-8aeb-4683951f7d45 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:27.531951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 650/1291] 10.222.0.24 () {56 vars in 1116 bytes} [Thu May 19 13:21:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:21:27.541712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:27.653097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4967c-865e-45d8-86a2-6a6f38526505 None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:27.688332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:27.688974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:27.689254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:27.689559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] PATH_INFO: `/v3/services` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:27.690266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:27.690575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:27.691944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb4729-d692-46ff-a688-8159d3fa0235 None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:27.696526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 642/1292] 10.222.0.24 () {56 vars in 1116 bytes} [Thu May 19 13:21:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:29.907260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1fe17-c4c9-40da-aab0-d19b38314ead None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:29.908110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1fe17-c4c9-40da-aab0-d19b38314ead None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:29.908110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1fe17-c4c9-40da-aab0-d19b38314ead None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:29.910336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 651/1293] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:29.919083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8d744-c3fc-427c-8c6e-6feac10ae3e0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:29.919545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8d744-c3fc-427c-8c6e-6feac10ae3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:29.919648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8d744-c3fc-427c-8c6e-6feac10ae3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:29.938381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dfa8d744-c3fc-427c-8c6e-6feac10ae3e0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:29.939572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dfa8d744-c3fc-427c-8c6e-6feac10ae3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:29.971353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:30.023142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 643/1294] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:30.046339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1fe17-c4c9-40da-aab0-d19b38314ead None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:30.081583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:30.159145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1fe17-c4c9-40da-aab0-d19b38314ead None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:30.206017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba0780-3727-4ba8-ad3e-09a2f7e53550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:30.206672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba0780-3727-4ba8-ad3e-09a2f7e53550 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:30.206954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba0780-3727-4ba8-ad3e-09a2f7e53550 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:30.207228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba0780-3727-4ba8-ad3e-09a2f7e53550 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:30.207939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba0780-3727-4ba8-ad3e-09a2f7e53550 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:30.209321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba0780-3727-4ba8-ad3e-09a2f7e53550 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:30.225261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 652/1295] 10.222.0.24 () {60 vars in 1141 bytes} [Thu May 19 13:21:30 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:21:32.438785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0204e-b745-42b1-936b-bbee2f3653cf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:32.439334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0204e-b745-42b1-936b-bbee2f3653cf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:32.439334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0204e-b745-42b1-936b-bbee2f3653cf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:32.440750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 644/1296] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:32.447199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:32.447878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:32.448231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:32.467896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:32.469116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:32.557479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 653/1297] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:32.587577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0204e-b745-42b1-936b-bbee2f3653cf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:32.703421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0204e-b745-42b1-936b-bbee2f3653cf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:32.744312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:32.745350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:32.745568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:32.745819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:32.746471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:32.747848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:32.760486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:32.762359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:32.763523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 645/1298] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:21:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:32.770280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:32.792110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:32.876891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bcc52-e54d-4959-9f69-dd8d021de13d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:32.914470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:32.915118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:32.915399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:32.915671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:32.916699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:32.917132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:32.918567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:32.923947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 654/1299] 10.222.0.24 () {56 vars in 1116 bytes} [Thu May 19 13:21:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:32.931812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:32.977864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:33.047072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877c5e0-4486-42c1-b767-6af7d6818e8c None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:33.092312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:33.092943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:33.093262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:33.093630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:33.094343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] RBAC: Adding query filter params (interface=public, service_id=11dbd1a508b246b498cb9c31708041d8, region_id=RegionOne) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:33.094778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:33.096020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bef13-5628-428d-a545-601c2638da8b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:33.101935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 646/1300] 10.222.0.24 () {56 vars in 1251 bytes} [Thu May 19 13:21:32 2022] GET /identity/v3/endpoints?service_id=11dbd1a508b246b498cb9c31708041d8&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:33.108337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:33.193319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:33.212594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07695200-5ebd-43d4-b17e-c7107b65fa07 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:33.253392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d2c29-8255-4071-9609-bf7675a93807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:33.254507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d2c29-8255-4071-9609-bf7675a93807 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:33.254968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d2c29-8255-4071-9609-bf7675a93807 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:33.255398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d2c29-8255-4071-9609-bf7675a93807 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:33.256375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d2c29-8255-4071-9609-bf7675a93807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:33.257813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d2c29-8255-4071-9609-bf7675a93807 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:33.266113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 655/1301] 10.222.0.24 () {56 vars in 1087 bytes} [Thu May 19 13:21:33 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:21:35.455626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dbfdd-a6f9-4f4e-8a62-58a3a2186086 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:35.456607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dbfdd-a6f9-4f4e-8a62-58a3a2186086 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:35.456607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dbfdd-a6f9-4f4e-8a62-58a3a2186086 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:35.458755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 647/1302] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:21:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:21:35.466919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:35.467373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:35.467472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:35.485725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:21:35.486900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:21:35.558516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 656/1303] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:21:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 13:21:35.582210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dbfdd-a6f9-4f4e-8a62-58a3a2186086 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:35.697327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:35.706420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dbfdd-a6f9-4f4e-8a62-58a3a2186086 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:35.742796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:35.743592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:35.743875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:35.744147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] PATH_INFO: `/v3/services/placement` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:35.744805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:35.746410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 13:21:35.758306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 13:21:35.759621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.get_or_create( May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application with Lock( May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self._enter() May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application return self.creator() May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application created_value = creator( May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: ERROR keystone.server.flask.application May 19 13:21:35.760582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 648/1304] 10.222.0.24 () {56 vars in 1117 bytes} [Thu May 19 13:21:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 13:21:35.766024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:35.845398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:35.879841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35f950-d66e-467d-ad98-7e57c14ece2e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:35.923589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:35.924524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:35.924908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:35.925275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:35.926373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:21:35.926805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:35.928952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0960ce-0f30-45ba-8dd1-3cead78b7fa6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:35.936159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 657/1305] 10.222.0.24 () {56 vars in 1116 bytes} [Thu May 19 13:21:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:21:35.942062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:21:36.047449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389cb2e-a17b-4be9-bd3e-eee7650c50ef None admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:21:36.096480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082d249-8379-4fde-911d-a2387c17867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:21:36.097102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d249-8379-4fde-911d-a2387c17867b None admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:21:36.097381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d249-8379-4fde-911d-a2387c17867b None admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:21:36.097687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082d249-8379-4fde-911d-a2387c17867b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:21:36.098427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082d249-8379-4fde-911d-a2387c17867b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:21:36.099793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082d249-8379-4fde-911d-a2387c17867b None admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:21:36.125773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 649/1306] 10.222.0.24 () {60 vars in 1144 bytes} [Thu May 19 13:21:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:21:45.932081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:21:46.120810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:40.290136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827075-288e-49c6-aaea-1ec633169ce5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:40.290584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827075-288e-49c6-aaea-1ec633169ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:40.290674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827075-288e-49c6-aaea-1ec633169ce5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:40.297883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874bbe5-54ed-4d57-8a9d-88bc72e350f9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:40.298340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874bbe5-54ed-4d57-8a9d-88bc72e350f9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:40.298424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874bbe5-54ed-4d57-8a9d-88bc72e350f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:40.301780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 658/1307] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:22:40 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:40.424171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f874bbe5-54ed-4d57-8a9d-88bc72e350f9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:40.428703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f874bbe5-54ed-4d57-8a9d-88bc72e350f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:40.618278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 650/1308] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:22:40.684008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b09b1-7527-4bc4-b944-794eae29255d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:40.684433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b09b1-7527-4bc4-b944-794eae29255d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:40.684812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b09b1-7527-4bc4-b944-794eae29255d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:40.686507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 659/1309] 10.222.0.24 () {54 vars in 964 bytes} [Thu May 19 13:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:40.694200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a96fb-9a92-40f9-86bd-063adb43bef4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:40.694672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a96fb-9a92-40f9-86bd-063adb43bef4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:40.695165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a96fb-9a92-40f9-86bd-063adb43bef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:40.797295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-613a96fb-9a92-40f9-86bd-063adb43bef4 None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:40.798691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-613a96fb-9a92-40f9-86bd-063adb43bef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:40.822403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:40.891413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 651/1310] 10.222.0.24 () {58 vars in 1061 bytes} [Thu May 19 13:22:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:40.899964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b09b1-7527-4bc4-b944-794eae29255d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:22:40.903147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3e6b09b1-7527-4bc4-b944-794eae29255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:41.031227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b09b1-7527-4bc4-b944-794eae29255d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:22:41.121150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:22:41.121877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:41.122273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:41.122550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:41.124208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:41.189622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:41.253065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:22:41.254733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b406a09-ca55-4b0a-9ea2-b47265194a0c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:22:41.276592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 660/1311] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:22:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:22:43.815536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a381-9ea2-44f5-a3b2-e97cb0bf68d0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:43.816088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a381-9ea2-44f5-a3b2-e97cb0bf68d0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:43.816088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872a381-9ea2-44f5-a3b2-e97cb0bf68d0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:43.817461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 652/1312] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:43.822949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e473d5a-092b-4c1b-8baa-41dc0ba9ad42 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:43.823456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e473d5a-092b-4c1b-8baa-41dc0ba9ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:43.823578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e473d5a-092b-4c1b-8baa-41dc0ba9ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:43.918687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6e473d5a-092b-4c1b-8baa-41dc0ba9ad42 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:43.920124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6e473d5a-092b-4c1b-8baa-41dc0ba9ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:44.045456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 661/1313] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:44.081946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872a381-9ea2-44f5-a3b2-e97cb0bf68d0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:22:44.084848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9872a381-9ea2-44f5-a3b2-e97cb0bf68d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:44.196049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872a381-9ea2-44f5-a3b2-e97cb0bf68d0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:22:44.274969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:44.278234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:22:44.278897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:44.279171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:44.279470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:44.281655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:44.391932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:22:44.393279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a0fb-2566-40bb-8344-473074edf4d5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:22:44.412037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 653/1314] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:22:46.912300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1893d-b702-4482-a990-bb65a6a6e8bc None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:46.912874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1893d-b702-4482-a990-bb65a6a6e8bc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:46.912874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1893d-b702-4482-a990-bb65a6a6e8bc None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:46.914249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 662/1315] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:46.921163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617705-4600-4507-b571-cdd620758537 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:46.921749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617705-4600-4507-b571-cdd620758537 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:46.921901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617705-4600-4507-b571-cdd620758537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:47.000127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-04617705-4600-4507-b571-cdd620758537 None None] MFA Rules not processed for user `b663cda428a149fa9a7a5c13fa738c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:47.001354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04617705-4600-4507-b571-cdd620758537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:47.060319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:47.106537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 654/1316] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:22:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:47.145540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1893d-b702-4482-a990-bb65a6a6e8bc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:22:47.147081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-83d1893d-b702-4482-a990-bb65a6a6e8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:47.157805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:47.259024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1893d-b702-4482-a990-bb65a6a6e8bc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:22:47.349534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:22:47.350196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:47.350481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:47.350762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:47.352420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:47.420143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:47.474359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:22:47.475815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23494e79-d637-4273-b4ca-b677ece5b56f service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:22:47.494669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 663/1317] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:22:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:22:50.322078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b3361-b1b9-4a35-81b9-df0ec7ee9e2a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:50.323090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b3361-b1b9-4a35-81b9-df0ec7ee9e2a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:50.323195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b3361-b1b9-4a35-81b9-df0ec7ee9e2a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:50.325313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 655/1318] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:22:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:50.333831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027378-a278-4bfa-985e-ca30dea5ad82 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:50.334295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027378-a278-4bfa-985e-ca30dea5ad82 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:50.334387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04027378-a278-4bfa-985e-ca30dea5ad82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:50.411895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-04027378-a278-4bfa-985e-ca30dea5ad82 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:50.413093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-04027378-a278-4bfa-985e-ca30dea5ad82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:50.526936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 664/1319] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:22:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:50.578085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80053bc3-3c2a-4f55-b7c1-171ab1f6e578 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:50.578345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80053bc3-3c2a-4f55-b7c1-171ab1f6e578 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:50.578650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80053bc3-3c2a-4f55-b7c1-171ab1f6e578 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:50.580044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 656/1320] 10.222.0.24 () {54 vars in 964 bytes} [Thu May 19 13:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:50.586958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef034f-8948-44ff-ab34-27462b370335 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:50.587271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef034f-8948-44ff-ab34-27462b370335 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:50.587592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef034f-8948-44ff-ab34-27462b370335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:50.614825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:50.672560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ffef034f-8948-44ff-ab34-27462b370335 None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:50.673786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ffef034f-8948-44ff-ab34-27462b370335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:50.757026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 665/1321] 10.222.0.24 () {58 vars in 1061 bytes} [Thu May 19 13:22:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:50.767313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80053bc3-3c2a-4f55-b7c1-171ab1f6e578 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:22:50.768864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80053bc3-3c2a-4f55-b7c1-171ab1f6e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:50.879030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80053bc3-3c2a-4f55-b7c1-171ab1f6e578 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:22:50.952766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:22:50.953430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:50.953796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:50.954073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:50.955711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:50.963946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:51.088034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:22:51.089519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365c2fc-373c-42e4-810a-99fcd48f8aaa service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:22:51.110358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 657/1322] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:22:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:22:54.474113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae666ef-60a7-4580-af38-011e47a1a55c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:54.474702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae666ef-60a7-4580-af38-011e47a1a55c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:54.474702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae666ef-60a7-4580-af38-011e47a1a55c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:54.476021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 666/1323] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:54.482304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9170-f703-47fa-9599-a86a79ee6206 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:54.482304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9170-f703-47fa-9599-a86a79ee6206 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:54.482741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9170-f703-47fa-9599-a86a79ee6206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:54.551869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-575b9170-f703-47fa-9599-a86a79ee6206 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:54.553008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-575b9170-f703-47fa-9599-a86a79ee6206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:54.613748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:54.654068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 658/1324] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:54.691572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae666ef-60a7-4580-af38-011e47a1a55c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:22:54.693227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bae666ef-60a7-4580-af38-011e47a1a55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:54.808005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae666ef-60a7-4580-af38-011e47a1a55c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:22:54.820786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:54.886583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:22:54.887235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:54.887515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:54.887793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:54.889847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:55.003549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:22:55.005009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894162bf-c92a-4a9d-a587-103eb4771a1d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:22:55.023743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 667/1325] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:22:57.329319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206e9bf-da24-48e5-a122-35bab171ce85 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:57.329940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206e9bf-da24-48e5-a122-35bab171ce85 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:57.329940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206e9bf-da24-48e5-a122-35bab171ce85 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:57.331438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 659/1326] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:22:57.336780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edde7c3-43c8-4485-b3c5-81c2b483d7e7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:57.337209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edde7c3-43c8-4485-b3c5-81c2b483d7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:57.337318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edde7c3-43c8-4485-b3c5-81c2b483d7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:57.390370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:57.409765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9edde7c3-43c8-4485-b3c5-81c2b483d7e7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:22:57.410971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9edde7c3-43c8-4485-b3c5-81c2b483d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:57.511704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 668/1327] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:22:57.547106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206e9bf-da24-48e5-a122-35bab171ce85 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:22:57.548661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4206e9bf-da24-48e5-a122-35bab171ce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:57.656220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206e9bf-da24-48e5-a122-35bab171ce85 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:22:57.732216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:22:57.732865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:22:57.733141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:22:57.733437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:22:57.735067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:22:57.752258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:22:57.846252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:22:57.847597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368fbde-948b-4445-b016-a00e5d37e6bc service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:22:57.866244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 660/1328] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:00.180058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dae15e-f4bb-411d-a20f-9501eb096d18 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:00.180657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dae15e-f4bb-411d-a20f-9501eb096d18 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:00.180657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dae15e-f4bb-411d-a20f-9501eb096d18 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:00.182475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 669/1329] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:00.188469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f2b7d-b0b3-4b10-98d1-2e20ff7f0964 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:00.188957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f2b7d-b0b3-4b10-98d1-2e20ff7f0964 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:00.189048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f2b7d-b0b3-4b10-98d1-2e20ff7f0964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:00.264373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7d1f2b7d-b0b3-4b10-98d1-2e20ff7f0964 None None] MFA Rules not processed for user `b663cda428a149fa9a7a5c13fa738c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:00.265687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d1f2b7d-b0b3-4b10-98d1-2e20ff7f0964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:00.335769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:00.369674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 661/1330] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:00.406500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dae15e-f4bb-411d-a20f-9501eb096d18 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:00.408133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f2dae15e-f4bb-411d-a20f-9501eb096d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:00.518797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dae15e-f4bb-411d-a20f-9501eb096d18 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:00.539997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:00.597080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:00.597770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:00.598056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:00.598335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:00.600020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:00.715717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:00.717167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe753-5b3b-49c3-97b2-53697f4bd121 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:00.736089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 670/1331] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:03.300869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a20599-c6ca-4ea8-a27b-fc8a15b84105 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:03.301490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a20599-c6ca-4ea8-a27b-fc8a15b84105 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:03.301605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a20599-c6ca-4ea8-a27b-fc8a15b84105 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:03.303485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 662/1332] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:03.310819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049f8b-5902-47c0-9143-d76df6183d75 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:03.311329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049f8b-5902-47c0-9143-d76df6183d75 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:03.311468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049f8b-5902-47c0-9143-d76df6183d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:03.379339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:03.382235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-10049f8b-5902-47c0-9143-d76df6183d75 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:03.384156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-10049f8b-5902-47c0-9143-d76df6183d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:03.489286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 671/1333] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:03.527031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a20599-c6ca-4ea8-a27b-fc8a15b84105 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:03.528684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a1a20599-c6ca-4ea8-a27b-fc8a15b84105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:03.643081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a20599-c6ca-4ea8-a27b-fc8a15b84105 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:03.722351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:03.723006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:03.723321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:03.723595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:03.725277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:03.755518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:03.848193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:03.849598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f61e5c-02ba-4d58-b4b2-d4965fbc0a33 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:03.869974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 663/1334] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:06.252768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de86a0-d75c-47e7-ab2a-738e0b43a57e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:06.254020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de86a0-d75c-47e7-ab2a-738e0b43a57e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:06.256208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de86a0-d75c-47e7-ab2a-738e0b43a57e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:06.257789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 672/1335] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:06.264176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ecde1-4438-4557-8dbe-3d9ac8cef762 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:06.264645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ecde1-4438-4557-8dbe-3d9ac8cef762 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:06.264740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ecde1-4438-4557-8dbe-3d9ac8cef762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:06.340881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e02ecde1-4438-4557-8dbe-3d9ac8cef762 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:06.342128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e02ecde1-4438-4557-8dbe-3d9ac8cef762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:06.418214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:06.443087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 664/1336] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:06.487609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de86a0-d75c-47e7-ab2a-738e0b43a57e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:06.489527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-56de86a0-d75c-47e7-ab2a-738e0b43a57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:06.599747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de86a0-d75c-47e7-ab2a-738e0b43a57e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:06.632844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:06.681548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:06.682199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:06.682484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:06.682782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:06.684433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:06.806237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:06.807766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fef5c-af83-42ff-abff-4f36d53adcdb service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:06.826514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 673/1337] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:09.481364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9abfb-95bd-4346-a957-8207e29b3725 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:09.482076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9abfb-95bd-4346-a957-8207e29b3725 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:09.482076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9abfb-95bd-4346-a957-8207e29b3725 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:09.483336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 665/1338] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:09.489797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e11d-bdc6-42a0-9ef2-b1fae28fbc41 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:09.490600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e11d-bdc6-42a0-9ef2-b1fae28fbc41 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:09.490778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06e11d-bdc6-42a0-9ef2-b1fae28fbc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:09.568126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ab06e11d-bdc6-42a0-9ef2-b1fae28fbc41 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:09.569342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ab06e11d-bdc6-42a0-9ef2-b1fae28fbc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:09.577947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:09.667982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 674/1339] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:09.708457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9abfb-95bd-4346-a957-8207e29b3725 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:09.711052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5b9abfb-95bd-4346-a957-8207e29b3725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:09.846980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b9abfb-95bd-4346-a957-8207e29b3725 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:09.922334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:09.922978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:09.923263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:09.923538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:09.925191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:09.966066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:10.060126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:10.062190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98151caf-24bc-4660-8f32-0df22f1b6303 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:10.081897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 666/1340] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:14.371133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287fc19-8930-498b-8c01-63c1b43dd2f7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:14.371657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287fc19-8930-498b-8c01-63c1b43dd2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:14.371657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287fc19-8930-498b-8c01-63c1b43dd2f7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:14.373201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 675/1341] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:14.378434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ad044-5b5b-48e0-a9dc-97e6a0cbffc3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:14.378869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ad044-5b5b-48e0-a9dc-97e6a0cbffc3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:14.378994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ad044-5b5b-48e0-a9dc-97e6a0cbffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:14.450359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-368ad044-5b5b-48e0-a9dc-97e6a0cbffc3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:14.451542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-368ad044-5b5b-48e0-a9dc-97e6a0cbffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:14.546603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:14.561037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 667/1342] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:14.600544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287fc19-8930-498b-8c01-63c1b43dd2f7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:14.603415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f287fc19-8930-498b-8c01-63c1b43dd2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:14.707903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287fc19-8930-498b-8c01-63c1b43dd2f7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:14.711590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcc385-0d21-44ca-ae07-1b3a36745066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:14.712306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcc385-0d21-44ca-ae07-1b3a36745066 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:14.712582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcc385-0d21-44ca-ae07-1b3a36745066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:14.712858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcc385-0d21-44ca-ae07-1b3a36745066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:14.722092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcc385-0d21-44ca-ae07-1b3a36745066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:14.723520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcc385-0d21-44ca-ae07-1b3a36745066 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:14.733046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 676/1343] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:17.508786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3e87a-2240-4696-85da-d1d62febc335 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:17.509989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3e87a-2240-4696-85da-d1d62febc335 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:17.509989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3e87a-2240-4696-85da-d1d62febc335 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:17.510687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 668/1344] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:23:17.516115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c22b-abc6-4ece-bf08-931b6befa522 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:17.516560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c22b-abc6-4ece-bf08-931b6befa522 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:17.516804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947c22b-abc6-4ece-bf08-931b6befa522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:17.545989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0947c22b-abc6-4ece-bf08-931b6befa522 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:17.547110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0947c22b-abc6-4ece-bf08-931b6befa522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:17.576169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:17.627928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 677/1345] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:17.664419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3e87a-2240-4696-85da-d1d62febc335 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:17.666839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d6e3e87a-2240-4696-85da-d1d62febc335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:17.680376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3e87a-2240-4696-85da-d1d62febc335 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:17.684035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382d886-65a3-42bc-bc5e-798925e8f1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:17.684675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d886-65a3-42bc-bc5e-798925e8f1ab service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:17.684957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d886-65a3-42bc-bc5e-798925e8f1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:17.685232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d886-65a3-42bc-bc5e-798925e8f1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:17.694241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382d886-65a3-42bc-bc5e-798925e8f1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:17.695660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382d886-65a3-42bc-bc5e-798925e8f1ab service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:17.705048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 669/1346] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:22.300093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de8aa-cc01-4ed8-a59d-99121326b5ea None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:22.300617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de8aa-cc01-4ed8-a59d-99121326b5ea None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:22.300617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13de8aa-cc01-4ed8-a59d-99121326b5ea None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:22.302049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 678/1347] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:22.307590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d199d9-33a2-4827-8c5e-21dfbbc1308b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:22.308054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d199d9-33a2-4827-8c5e-21dfbbc1308b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:22.308167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d199d9-33a2-4827-8c5e-21dfbbc1308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:22.326898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d1d199d9-33a2-4827-8c5e-21dfbbc1308b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:22.328088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d1d199d9-33a2-4827-8c5e-21dfbbc1308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:22.336441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 670/1348] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:22.373865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de8aa-cc01-4ed8-a59d-99121326b5ea None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:22.385271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13de8aa-cc01-4ed8-a59d-99121326b5ea None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:22.389135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb92847-b91c-4497-beb9-eff6f3425875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:22.389802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92847-b91c-4497-beb9-eff6f3425875 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:22.390087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92847-b91c-4497-beb9-eff6f3425875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:22.390360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92847-b91c-4497-beb9-eff6f3425875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:22.403905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb92847-b91c-4497-beb9-eff6f3425875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:22.405339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb92847-b91c-4497-beb9-eff6f3425875 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:22.415129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 679/1349] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:27.834050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fb7fc-abe8-49bc-b948-df653015e11c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:27.835047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fb7fc-abe8-49bc-b948-df653015e11c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:27.835517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fb7fc-abe8-49bc-b948-df653015e11c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:27.837520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 671/1350] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:23:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:27.842850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a24e96-b24f-4660-93a7-d9fd81efd817 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:27.843285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a24e96-b24f-4660-93a7-d9fd81efd817 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:27.843398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a24e96-b24f-4660-93a7-d9fd81efd817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:27.916279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c3a24e96-b24f-4660-93a7-d9fd81efd817 None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:27.918083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c3a24e96-b24f-4660-93a7-d9fd81efd817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:28.017439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 680/1351] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:23:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:32.321553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=13 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:32.328254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91391-a86d-42b4-8d21-a7863113d3dc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:32.329373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91391-a86d-42b4-8d21-a7863113d3dc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:32.330187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91391-a86d-42b4-8d21-a7863113d3dc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:32.332356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 672/1352] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:23:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:23:32.337853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725ed6-5aaf-4627-bb69-e73dcaa0871d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:32.338742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725ed6-5aaf-4627-bb69-e73dcaa0871d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:32.339243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725ed6-5aaf-4627-bb69-e73dcaa0871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:32.363799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-73725ed6-5aaf-4627-bb69-e73dcaa0871d None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:32.365793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73725ed6-5aaf-4627-bb69-e73dcaa0871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:32.376431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 681/1353] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:23:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:33.785119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072c5b6-f903-4747-a73d-7fc9d87a410d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:33.785743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072c5b6-f903-4747-a73d-7fc9d87a410d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:33.785743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072c5b6-f903-4747-a73d-7fc9d87a410d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:33.787293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 673/1354] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:33.792928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4710-56c9-42c8-99bf-27d518990f3c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:33.793373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4710-56c9-42c8-99bf-27d518990f3c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:33.793636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4710-56c9-42c8-99bf-27d518990f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:33.814733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-781b4710-56c9-42c8-99bf-27d518990f3c None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:33.815983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-781b4710-56c9-42c8-99bf-27d518990f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:33.824836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 682/1355] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:23:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:37.902062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20bad8-6c04-40a0-9400-39a538c6d8c4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:37.902605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20bad8-6c04-40a0-9400-39a538c6d8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:37.902605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20bad8-6c04-40a0-9400-39a538c6d8c4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:37.903950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 674/1356] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:37.909276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1136a7-2d0b-49e5-b8c9-34097e618284 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:37.909737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1136a7-2d0b-49e5-b8c9-34097e618284 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:37.909819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1136a7-2d0b-49e5-b8c9-34097e618284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:37.928406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fb1136a7-2d0b-49e5-b8c9-34097e618284 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:37.929627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fb1136a7-2d0b-49e5-b8c9-34097e618284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:37.938816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 683/1357] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:38.013115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6cf06-d882-4978-91b3-36f03267b829 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:38.014294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6cf06-d882-4978-91b3-36f03267b829 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:38.014733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6cf06-d882-4978-91b3-36f03267b829 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:38.016557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 675/1358] 10.222.0.24 () {54 vars in 961 bytes} [Thu May 19 13:23:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:38.022169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce46d1-b616-47fc-af2f-808e9b8215d6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:38.022459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce46d1-b616-47fc-af2f-808e9b8215d6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:38.022742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce46d1-b616-47fc-af2f-808e9b8215d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:38.075655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:38.085967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4bce46d1-b616-47fc-af2f-808e9b8215d6 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:38.088366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4bce46d1-b616-47fc-af2f-808e9b8215d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:38.206533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 684/1359] 10.222.0.24 () {58 vars in 1058 bytes} [Thu May 19 13:23:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:38.215996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6cf06-d882-4978-91b3-36f03267b829 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:38.232612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6cf06-d882-4978-91b3-36f03267b829 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:38.254180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce66087-8311-49de-9b64-69c123993770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:38.262939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce66087-8311-49de-9b64-69c123993770 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:38.263829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce66087-8311-49de-9b64-69c123993770 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:38.264270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce66087-8311-49de-9b64-69c123993770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:38.277151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce66087-8311-49de-9b64-69c123993770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:38.279053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce66087-8311-49de-9b64-69c123993770 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:38.293689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 676/1360] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:40.817501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8af7b-00f7-46b3-a7b3-d4aeffb818d8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:40.818048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8af7b-00f7-46b3-a7b3-d4aeffb818d8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:40.818048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8af7b-00f7-46b3-a7b3-d4aeffb818d8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:40.819384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 685/1361] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:40.825378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dbe78-4fb2-4fb0-aa99-6e3e3b04e900 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:40.825859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dbe78-4fb2-4fb0-aa99-6e3e3b04e900 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:40.825968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7dbe78-4fb2-4fb0-aa99-6e3e3b04e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:40.845288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4a7dbe78-4fb2-4fb0-aa99-6e3e3b04e900 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:40.846530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4a7dbe78-4fb2-4fb0-aa99-6e3e3b04e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:40.854831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 677/1362] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:40.898864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8af7b-00f7-46b3-a7b3-d4aeffb818d8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:40.916831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8af7b-00f7-46b3-a7b3-d4aeffb818d8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:40.923400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8640a44-92ea-4a2a-b047-df581cf516e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:40.924229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8640a44-92ea-4a2a-b047-df581cf516e0 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:40.924676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8640a44-92ea-4a2a-b047-df581cf516e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:40.925108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8640a44-92ea-4a2a-b047-df581cf516e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:40.935092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8640a44-92ea-4a2a-b047-df581cf516e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:40.936935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8640a44-92ea-4a2a-b047-df581cf516e0 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:40.948246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 686/1363] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:43.482292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd65cc-44df-44a8-bbbb-6a970b02e989 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:43.483376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd65cc-44df-44a8-bbbb-6a970b02e989 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:43.483845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd65cc-44df-44a8-bbbb-6a970b02e989 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:43.485380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 678/1364] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:43.490978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390814ad-2221-4b4b-aff8-941456331fbc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:43.491439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390814ad-2221-4b4b-aff8-941456331fbc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:43.491544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390814ad-2221-4b4b-aff8-941456331fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:43.513888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-390814ad-2221-4b4b-aff8-941456331fbc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:43.522792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-390814ad-2221-4b4b-aff8-941456331fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:43.534529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 687/1365] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:23:43.582305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab2de-58b1-4228-bd6c-d1426086e67c None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:43.582596 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab2de-58b1-4228-bd6c-d1426086e67c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:43.582899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab2de-58b1-4228-bd6c-d1426086e67c None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:43.584342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 679/1366] 10.222.0.24 () {54 vars in 961 bytes} [Thu May 19 13:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:43.590495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a630a-6150-4799-97ad-8ec5f72b2db9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:43.590782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a630a-6150-4799-97ad-8ec5f72b2db9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:43.591061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a630a-6150-4799-97ad-8ec5f72b2db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:43.609337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e77a630a-6150-4799-97ad-8ec5f72b2db9 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:43.610545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e77a630a-6150-4799-97ad-8ec5f72b2db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:43.620203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 688/1367] 10.222.0.24 () {58 vars in 1058 bytes} [Thu May 19 13:23:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:43.628273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab2de-58b1-4228-bd6c-d1426086e67c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:43.644268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab2de-58b1-4228-bd6c-d1426086e67c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:43.650618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812191d1-593c-43fc-8e3d-c1c6030c2fa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:43.651252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812191d1-593c-43fc-8e3d-c1c6030c2fa9 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:43.651540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812191d1-593c-43fc-8e3d-c1c6030c2fa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:43.651819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812191d1-593c-43fc-8e3d-c1c6030c2fa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:43.660339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812191d1-593c-43fc-8e3d-c1c6030c2fa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:43.661950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812191d1-593c-43fc-8e3d-c1c6030c2fa9 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:43.672022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 680/1368] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:45.946590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27cb72-bc26-4fbc-b777-b7c127f4c50a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:45.947498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27cb72-bc26-4fbc-b777-b7c127f4c50a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:45.947736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27cb72-bc26-4fbc-b777-b7c127f4c50a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:45.949283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 689/1369] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:45.954805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30812c81-0759-4b14-948b-cd8d4f849af7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:45.955280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30812c81-0759-4b14-948b-cd8d4f849af7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:45.955403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30812c81-0759-4b14-948b-cd8d4f849af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:45.973592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-30812c81-0759-4b14-948b-cd8d4f849af7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:45.974792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-30812c81-0759-4b14-948b-cd8d4f849af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:45.983157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 681/1370] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:23:46.026790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27cb72-bc26-4fbc-b777-b7c127f4c50a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:46.043934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27cb72-bc26-4fbc-b777-b7c127f4c50a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:46.050744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351dca1-5e2d-4715-9215-f95db58532f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:46.051481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351dca1-5e2d-4715-9215-f95db58532f4 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:46.051770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351dca1-5e2d-4715-9215-f95db58532f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:46.051970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351dca1-5e2d-4715-9215-f95db58532f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:46.061855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351dca1-5e2d-4715-9215-f95db58532f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:46.063466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351dca1-5e2d-4715-9215-f95db58532f4 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:46.074172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 690/1371] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:46.230892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa9064-ae46-42f8-80f4-403fd152fc88 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:46.231130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa9064-ae46-42f8-80f4-403fd152fc88 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:46.231318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa9064-ae46-42f8-80f4-403fd152fc88 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:46.232973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 682/1372] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:46.241193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb61606-a90b-440b-a85b-49da8353d1cc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:46.241865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb61606-a90b-440b-a85b-49da8353d1cc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:46.242418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb61606-a90b-440b-a85b-49da8353d1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:46.266718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7cb61606-a90b-440b-a85b-49da8353d1cc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:46.268151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7cb61606-a90b-440b-a85b-49da8353d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:46.277972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 691/1373] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:46.590309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572fc67-945c-4718-ba32-e5df17357c6f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:46.591305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572fc67-945c-4718-ba32-e5df17357c6f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:46.591305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572fc67-945c-4718-ba32-e5df17357c6f None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:46.592629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 683/1374] 10.222.0.24 () {54 vars in 963 bytes} [Thu May 19 13:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:46.597743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fabee-7b9e-4e1c-a870-5d40b601c429 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:46.598103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fabee-7b9e-4e1c-a870-5d40b601c429 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:46.598449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fabee-7b9e-4e1c-a870-5d40b601c429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:46.652897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-371fabee-7b9e-4e1c-a870-5d40b601c429 None None] MFA Rules not processed for user `d92108b779ca47aca8f941b7dfae80d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:46.654109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-371fabee-7b9e-4e1c-a870-5d40b601c429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:46.718490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 692/1375] 10.222.0.24 () {58 vars in 1060 bytes} [Thu May 19 13:23:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:46.726079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572fc67-945c-4718-ba32-e5df17357c6f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:46.738578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572fc67-945c-4718-ba32-e5df17357c6f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:46.742883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df10a5-8159-4b15-b267-3d25f224229f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:46.744007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df10a5-8159-4b15-b267-3d25f224229f service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:46.744726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df10a5-8159-4b15-b267-3d25f224229f service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:46.745211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df10a5-8159-4b15-b267-3d25f224229f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:46.755896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df10a5-8159-4b15-b267-3d25f224229f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:46.757774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df10a5-8159-4b15-b267-3d25f224229f service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:46.769085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 684/1376] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:48.339720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b660-94a1-4189-8189-1051aafa20df None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:48.340674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b660-94a1-4189-8189-1051aafa20df None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:48.340962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b660-94a1-4189-8189-1051aafa20df None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:48.342477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 693/1377] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:48.348806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75a79-4a8a-412d-baa2-5932a0c4215c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:48.349351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75a79-4a8a-412d-baa2-5932a0c4215c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:48.349351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a75a79-4a8a-412d-baa2-5932a0c4215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:48.369900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d9a75a79-4a8a-412d-baa2-5932a0c4215c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:48.371182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d9a75a79-4a8a-412d-baa2-5932a0c4215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:48.384968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 685/1378] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:48.431200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b660-94a1-4189-8189-1051aafa20df None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:48.447666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b660-94a1-4189-8189-1051aafa20df None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:48.453705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23e554-7954-4d3c-a64d-bd102528a1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:48.454561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23e554-7954-4d3c-a64d-bd102528a1e4 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:48.454850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23e554-7954-4d3c-a64d-bd102528a1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:48.455135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23e554-7954-4d3c-a64d-bd102528a1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:48.464945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23e554-7954-4d3c-a64d-bd102528a1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:48.466621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23e554-7954-4d3c-a64d-bd102528a1e4 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:48.470986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:23:48.478796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 694/1379] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:50.725346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14738864-c4af-40fa-a68a-2fe32c158c95 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:50.725975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14738864-c4af-40fa-a68a-2fe32c158c95 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:50.725975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14738864-c4af-40fa-a68a-2fe32c158c95 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:50.727471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 686/1380] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:50.733166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cbca6-589b-49a1-9dfe-e2f4a212d99f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:50.733644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cbca6-589b-49a1-9dfe-e2f4a212d99f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:50.733899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cbca6-589b-49a1-9dfe-e2f4a212d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:50.753075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d28cbca6-589b-49a1-9dfe-e2f4a212d99f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:50.754303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d28cbca6-589b-49a1-9dfe-e2f4a212d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:50.762431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 695/1381] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:50.807316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14738864-c4af-40fa-a68a-2fe32c158c95 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:50.823293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14738864-c4af-40fa-a68a-2fe32c158c95 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:50.828984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fce50-4aef-4e20-a2b0-6b26b4bf00fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:50.829691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fce50-4aef-4e20-a2b0-6b26b4bf00fd service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:50.829976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fce50-4aef-4e20-a2b0-6b26b4bf00fd service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:50.830252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fce50-4aef-4e20-a2b0-6b26b4bf00fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:50.839946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fce50-4aef-4e20-a2b0-6b26b4bf00fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:50.841579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fce50-4aef-4e20-a2b0-6b26b4bf00fd service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:50.852747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 687/1382] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:53.126464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f56a2-8521-4601-b1bc-b5308054051a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:53.126979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f56a2-8521-4601-b1bc-b5308054051a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:53.126979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f56a2-8521-4601-b1bc-b5308054051a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:53.128491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 696/1383] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:53.134142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9c3b0-35c1-476d-8c3d-4753e5178004 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:53.134584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9c3b0-35c1-476d-8c3d-4753e5178004 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:53.134757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9c3b0-35c1-476d-8c3d-4753e5178004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:53.153894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c8a9c3b0-35c1-476d-8c3d-4753e5178004 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:53.155105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c8a9c3b0-35c1-476d-8c3d-4753e5178004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:53.163763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 688/1384] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:53.205102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f56a2-8521-4601-b1bc-b5308054051a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:53.220577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f56a2-8521-4601-b1bc-b5308054051a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:53.226140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471cde2-1b97-4826-bffd-7f62f8af4670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:53.227038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471cde2-1b97-4826-bffd-7f62f8af4670 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:53.227326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471cde2-1b97-4826-bffd-7f62f8af4670 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:53.227600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471cde2-1b97-4826-bffd-7f62f8af4670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:53.236173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471cde2-1b97-4826-bffd-7f62f8af4670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:53.241170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c471cde2-1b97-4826-bffd-7f62f8af4670 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:53.249598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 697/1385] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:55.451233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b13f6-cb4e-4c24-bc49-f4290d44ba96 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:55.451801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b13f6-cb4e-4c24-bc49-f4290d44ba96 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:55.451801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b13f6-cb4e-4c24-bc49-f4290d44ba96 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:55.453269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 689/1386] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:55.460026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a64e9-f4a4-4b9f-a123-c261847f3a77 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:55.460557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a64e9-f4a4-4b9f-a123-c261847f3a77 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:55.461023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a64e9-f4a4-4b9f-a123-c261847f3a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:55.488061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-063a64e9-f4a4-4b9f-a123-c261847f3a77 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:55.489924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-063a64e9-f4a4-4b9f-a123-c261847f3a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:55.502333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 698/1387] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:55.547484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b13f6-cb4e-4c24-bc49-f4290d44ba96 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:55.564248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b13f6-cb4e-4c24-bc49-f4290d44ba96 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:55.571556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922f98f-c91a-4cc0-a980-4c6e536ad1d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:55.572238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922f98f-c91a-4cc0-a980-4c6e536ad1d9 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:55.572516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922f98f-c91a-4cc0-a980-4c6e536ad1d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:55.572853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922f98f-c91a-4cc0-a980-4c6e536ad1d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:55.581707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922f98f-c91a-4cc0-a980-4c6e536ad1d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:55.583322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922f98f-c91a-4cc0-a980-4c6e536ad1d9 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:55.593176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 690/1388] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:23:57.827774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafd93e-28c9-4c9b-a13f-d42db2b98480 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:57.828996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafd93e-28c9-4c9b-a13f-d42db2b98480 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:57.829339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafd93e-28c9-4c9b-a13f-d42db2b98480 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:57.831296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 699/1389] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:23:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:23:57.836837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b659-74e6-4eea-b696-d506c24b267a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:57.837463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b659-74e6-4eea-b696-d506c24b267a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:57.837463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b659-74e6-4eea-b696-d506c24b267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:57.855404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3045b659-74e6-4eea-b696-d506c24b267a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:23:57.856618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3045b659-74e6-4eea-b696-d506c24b267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:23:57.864614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 691/1390] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:23:57.908884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafd93e-28c9-4c9b-a13f-d42db2b98480 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:23:57.924265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafd93e-28c9-4c9b-a13f-d42db2b98480 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:23:57.930064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c302a7-330d-4da7-9134-8e9bd6bef6b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:23:57.930687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c302a7-330d-4da7-9134-8e9bd6bef6b4 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:23:57.931000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c302a7-330d-4da7-9134-8e9bd6bef6b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:23:57.931231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c302a7-330d-4da7-9134-8e9bd6bef6b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:23:57.939829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c302a7-330d-4da7-9134-8e9bd6bef6b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:23:57.941457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c302a7-330d-4da7-9134-8e9bd6bef6b4 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:23:57.951270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 700/1391] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:24:00.208736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b316138-523f-4dd8-b96e-5ec84764dcff None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:00.209298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b316138-523f-4dd8-b96e-5ec84764dcff None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:00.209298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b316138-523f-4dd8-b96e-5ec84764dcff None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:00.210937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 692/1392] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:00.216928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b3e8c-b303-42a8-9ac7-69ea11d3a98c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:00.217384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b3e8c-b303-42a8-9ac7-69ea11d3a98c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:00.217529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b3e8c-b303-42a8-9ac7-69ea11d3a98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:00.239581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7a3b3e8c-b303-42a8-9ac7-69ea11d3a98c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:00.241881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a3b3e8c-b303-42a8-9ac7-69ea11d3a98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:00.255042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 701/1393] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:00.304350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b316138-523f-4dd8-b96e-5ec84764dcff None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:00.320182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b316138-523f-4dd8-b96e-5ec84764dcff None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:00.325953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0edfc-4a98-4df3-952d-860cbbcd4fa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:00.326627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0edfc-4a98-4df3-952d-860cbbcd4fa6 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:00.326908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0edfc-4a98-4df3-952d-860cbbcd4fa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:00.327182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0edfc-4a98-4df3-952d-860cbbcd4fa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:00.336256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0edfc-4a98-4df3-952d-860cbbcd4fa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:00.338020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0edfc-4a98-4df3-952d-860cbbcd4fa6 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:00.352311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 693/1394] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:02.623994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d678c2-cd50-483f-9235-840fd1610e35 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.624598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d678c2-cd50-483f-9235-840fd1610e35 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.624598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d678c2-cd50-483f-9235-840fd1610e35 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.625950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 702/1395] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:02.631310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51308-d08f-4433-b502-a25142e35a12 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.631814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51308-d08f-4433-b502-a25142e35a12 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.631814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51308-d08f-4433-b502-a25142e35a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.638835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=52 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:24:02.652439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-98d51308-d08f-4433-b502-a25142e35a12 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:02.653708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-98d51308-d08f-4433-b502-a25142e35a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:02.662281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 694/1396] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:02.698303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81f32c-fd4f-45a0-abe7-44d56bf32aa0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.698591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81f32c-fd4f-45a0-abe7-44d56bf32aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.698871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81f32c-fd4f-45a0-abe7-44d56bf32aa0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.700399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 703/1397] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:02.706059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56d96-e013-4526-952a-a89928cc206b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.706310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56d96-e013-4526-952a-a89928cc206b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.706579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd56d96-e013-4526-952a-a89928cc206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.708656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81f32c-fd4f-45a0-abe7-44d56bf32aa0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:02.722869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81f32c-fd4f-45a0-abe7-44d56bf32aa0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:02.725083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6fd56d96-e013-4526-952a-a89928cc206b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:02.726513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6fd56d96-e013-4526-952a-a89928cc206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:02.728383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6983664-df23-4c4b-adc3-1419abdb86e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:02.729044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6983664-df23-4c4b-adc3-1419abdb86e8 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.729310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6983664-df23-4c4b-adc3-1419abdb86e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.729695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6983664-df23-4c4b-adc3-1419abdb86e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.735114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 695/1398] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:24:02.738017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6983664-df23-4c4b-adc3-1419abdb86e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:02.739602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6983664-df23-4c4b-adc3-1419abdb86e8 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:02.749059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 704/1399] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:02.791365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4608d-a589-4c1b-9be0-b24975b9fd0a None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.791654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4608d-a589-4c1b-9be0-b24975b9fd0a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.791936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4608d-a589-4c1b-9be0-b24975b9fd0a None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.793318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 696/1400] 10.222.0.24 () {54 vars in 962 bytes} [Thu May 19 13:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:24:02.798380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad12ab-5af9-4e27-b2cf-3018dd13ac61 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.798667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad12ab-5af9-4e27-b2cf-3018dd13ac61 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.798945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad12ab-5af9-4e27-b2cf-3018dd13ac61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.854832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d5ad12ab-5af9-4e27-b2cf-3018dd13ac61 None None] MFA Rules not processed for user `7ea7d2d4fd034e66aa589875e3e27ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:02.856043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d5ad12ab-5af9-4e27-b2cf-3018dd13ac61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:02.926509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 705/1401] 10.222.0.24 () {58 vars in 1059 bytes} [Thu May 19 13:24:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:24:02.934571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4608d-a589-4c1b-9be0-b24975b9fd0a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:02.946212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4608d-a589-4c1b-9be0-b24975b9fd0a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:02.949943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1f17c-3fb1-4c2a-9ad4-e89ad2e6b612 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:02.950601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f17c-3fb1-4c2a-9ad4-e89ad2e6b612 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:02.950887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f17c-3fb1-4c2a-9ad4-e89ad2e6b612 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:02.951164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1f17c-3fb1-4c2a-9ad4-e89ad2e6b612 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:02.961955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1f17c-3fb1-4c2a-9ad4-e89ad2e6b612 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:02.963437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1f17c-3fb1-4c2a-9ad4-e89ad2e6b612 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:02.974219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 697/1402] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:03.528175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e0d8-b239-4082-ad3c-af7beb03bab6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:03.528629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e0d8-b239-4082-ad3c-af7beb03bab6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:03.528916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0e0d8-b239-4082-ad3c-af7beb03bab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:03.581176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-52d0e0d8-b239-4082-ad3c-af7beb03bab6 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:03.582492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-52d0e0d8-b239-4082-ad3c-af7beb03bab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:03.628455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:24:03.648748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 706/1403] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:24:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:03.691018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660828d-0ad9-474c-805c-e050b6e28a5d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:03.691550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660828d-0ad9-474c-805c-e050b6e28a5d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:03.691550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660828d-0ad9-474c-805c-e050b6e28a5d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:03.692977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 698/1404] 10.222.0.24 () {54 vars in 942 bytes} [Thu May 19 13:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:03.700109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ea2c6-dd22-4dcc-9515-8212b3130867 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:03.700402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ea2c6-dd22-4dcc-9515-8212b3130867 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:03.700683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ea2c6-dd22-4dcc-9515-8212b3130867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:03.750480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2d3ea2c6-dd22-4dcc-9515-8212b3130867 None None] MFA Rules not processed for user `78b2f9c966e24c46a23631b80a578862`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:03.751696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2d3ea2c6-dd22-4dcc-9515-8212b3130867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:03.828252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 707/1405] 10.222.0.24 () {58 vars in 1039 bytes} [Thu May 19 13:24:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:03.837850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660828d-0ad9-474c-805c-e050b6e28a5d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:03.850953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660828d-0ad9-474c-805c-e050b6e28a5d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:03.855930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad45fb0-3085-4e5d-9540-650b6d0d1278 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:03.856562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45fb0-3085-4e5d-9540-650b6d0d1278 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:03.856841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45fb0-3085-4e5d-9540-650b6d0d1278 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:03.857115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45fb0-3085-4e5d-9540-650b6d0d1278 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:03.865136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad45fb0-3085-4e5d-9540-650b6d0d1278 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:03.866855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad45fb0-3085-4e5d-9540-650b6d0d1278 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:03.874788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 699/1406] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:24:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:24:04.985350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73fe2-34a4-4d17-939c-20f6d673b6b8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:04.985934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73fe2-34a4-4d17-939c-20f6d673b6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:04.985934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73fe2-34a4-4d17-939c-20f6d673b6b8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:04.988340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 708/1407] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:04.992820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7e502-f541-4005-b59d-3ce109eb7cf6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:04.993313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7e502-f541-4005-b59d-3ce109eb7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:04.993452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7e502-f541-4005-b59d-3ce109eb7cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:05.011600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-09e7e502-f541-4005-b59d-3ce109eb7cf6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:05.013041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-09e7e502-f541-4005-b59d-3ce109eb7cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:05.021155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 700/1408] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:24:05.073908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed73fe2-34a4-4d17-939c-20f6d673b6b8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:05.100371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed73fe2-34a4-4d17-939c-20f6d673b6b8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:05.112178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca24c3-191f-45c0-b732-6b35e11bc0ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:05.113104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca24c3-191f-45c0-b732-6b35e11bc0ab service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:05.113614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca24c3-191f-45c0-b732-6b35e11bc0ab service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:05.114054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca24c3-191f-45c0-b732-6b35e11bc0ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:05.129306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca24c3-191f-45c0-b732-6b35e11bc0ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:05.134828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca24c3-191f-45c0-b732-6b35e11bc0ab service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:05.149600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 709/1409] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:15.006647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=20 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:24:15.139748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=56 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:24:26.607476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a84d1-bf3f-4820-a784-68e28f14f3f1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:26.608148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a84d1-bf3f-4820-a784-68e28f14f3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:26.608148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a84d1-bf3f-4820-a784-68e28f14f3f1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:26.609454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 701/1410] 10.222.0.24 () {54 vars in 918 bytes} [Thu May 19 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:26.615596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9d9e7-d582-4d57-b01c-3673c1c66244 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:26.616153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9d9e7-d582-4d57-b01c-3673c1c66244 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:26.616153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9d9e7-d582-4d57-b01c-3673c1c66244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:26.637733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6bb9d9e7-d582-4d57-b01c-3673c1c66244 None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:26.638922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6bb9d9e7-d582-4d57-b01c-3673c1c66244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:26.648286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 710/1411] 10.222.0.24 () {58 vars in 1015 bytes} [Thu May 19 13:24:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:26.662710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25598478-e963-46f4-a9ab-fc51ea56e1c7 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:26.663007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25598478-e963-46f4-a9ab-fc51ea56e1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:26.663487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25598478-e963-46f4-a9ab-fc51ea56e1c7 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:26.664864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 702/1412] 10.222.0.24 () {54 vars in 918 bytes} [Thu May 19 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:26.669803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb24a9-6d4f-4e5b-822d-f845cf4921c6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:26.670091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb24a9-6d4f-4e5b-822d-f845cf4921c6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:26.670361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb24a9-6d4f-4e5b-822d-f845cf4921c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:26.689472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b0fb24a9-6d4f-4e5b-822d-f845cf4921c6 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:26.690667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0fb24a9-6d4f-4e5b-822d-f845cf4921c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:26.700615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 711/1413] 10.222.0.24 () {58 vars in 1015 bytes} [Thu May 19 13:24:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:29.374600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0b1a-a6aa-48bc-a2f8-2acfc92d188b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:29.375500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0b1a-a6aa-48bc-a2f8-2acfc92d188b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:29.375500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0b1a-a6aa-48bc-a2f8-2acfc92d188b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:29.377803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:29.383420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db023e-1580-4adc-92fb-a744c526ec1f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:29.383907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db023e-1580-4adc-92fb-a744c526ec1f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:29.383907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db023e-1580-4adc-92fb-a744c526ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:29.402997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-24db023e-1580-4adc-92fb-a744c526ec1f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:29.404181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-24db023e-1580-4adc-92fb-a744c526ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:29.413293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:29.459130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb0b1a-a6aa-48bc-a2f8-2acfc92d188b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:29.476789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb0b1a-a6aa-48bc-a2f8-2acfc92d188b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:29.482579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93030eab-7fe4-4451-a148-6ef03db4075d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:29.483220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93030eab-7fe4-4451-a148-6ef03db4075d service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:29.483508 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93030eab-7fe4-4451-a148-6ef03db4075d service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:29.483785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93030eab-7fe4-4451-a148-6ef03db4075d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:29.492409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93030eab-7fe4-4451-a148-6ef03db4075d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:29.494073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93030eab-7fe4-4451-a148-6ef03db4075d service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:29.503664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 704/1416] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:30.515859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b97b-11cf-43b6-93b6-378c8317e180 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:30.516419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b97b-11cf-43b6-93b6-378c8317e180 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:30.516419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373b97b-11cf-43b6-93b6-378c8317e180 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:30.517741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:30.524230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cc4d-b119-49a5-a4dc-07fa9c1fcd44 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:30.524691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cc4d-b119-49a5-a4dc-07fa9c1fcd44 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:30.524804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cc4d-b119-49a5-a4dc-07fa9c1fcd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:30.546939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3e87cc4d-b119-49a5-a4dc-07fa9c1fcd44 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:30.548167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3e87cc4d-b119-49a5-a4dc-07fa9c1fcd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:30.557616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:30.722292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b97b-11cf-43b6-93b6-378c8317e180 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:30.745017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373b97b-11cf-43b6-93b6-378c8317e180 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:30.755669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ab632-0e20-4c84-b870-14216cf83b5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:30.756583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ab632-0e20-4c84-b870-14216cf83b5e service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:30.756859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ab632-0e20-4c84-b870-14216cf83b5e service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:30.757128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ab632-0e20-4c84-b870-14216cf83b5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:30.765808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ab632-0e20-4c84-b870-14216cf83b5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:30.767459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ab632-0e20-4c84-b870-14216cf83b5e service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:30.776847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 714/1419] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:31.737196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40394cee-bbfe-455c-9c2e-2743e1f5807e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:31.738124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40394cee-bbfe-455c-9c2e-2743e1f5807e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:31.738124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40394cee-bbfe-455c-9c2e-2743e1f5807e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:31.740255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:31.745732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaddbc-cd3f-40c0-bed0-6f3957a5cad5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:31.746186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaddbc-cd3f-40c0-bed0-6f3957a5cad5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:31.746306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbaddbc-cd3f-40c0-bed0-6f3957a5cad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:31.765994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7cbaddbc-cd3f-40c0-bed0-6f3957a5cad5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:31.767194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7cbaddbc-cd3f-40c0-bed0-6f3957a5cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:31.776373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:31.925322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40394cee-bbfe-455c-9c2e-2743e1f5807e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:31.948292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40394cee-bbfe-455c-9c2e-2743e1f5807e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:31.954869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02f9ef-6dcf-4233-80b8-697d3a36d338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:31.955524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02f9ef-6dcf-4233-80b8-697d3a36d338 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:31.955809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02f9ef-6dcf-4233-80b8-697d3a36d338 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:31.956078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02f9ef-6dcf-4233-80b8-697d3a36d338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:31.964583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02f9ef-6dcf-4233-80b8-697d3a36d338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:31.966236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02f9ef-6dcf-4233-80b8-697d3a36d338 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:31.975943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 707/1422] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:32.914733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bb100-4186-4e98-9497-5cbe26027dfe None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:32.915925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bb100-4186-4e98-9497-5cbe26027dfe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:32.915925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bb100-4186-4e98-9497-5cbe26027dfe None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:32.918165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:32.926674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166cc0-95a0-491e-afda-9c1af8fa4520 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:32.927363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166cc0-95a0-491e-afda-9c1af8fa4520 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:32.927581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166cc0-95a0-491e-afda-9c1af8fa4520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:32.954144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-73166cc0-95a0-491e-afda-9c1af8fa4520 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:32.955345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-73166cc0-95a0-491e-afda-9c1af8fa4520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:32.965318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:33.124017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bb100-4186-4e98-9497-5cbe26027dfe None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:33.143537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bb100-4186-4e98-9497-5cbe26027dfe None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:33.149945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cee5d2-c604-4bc7-838a-201ad52c59fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:33.150588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cee5d2-c604-4bc7-838a-201ad52c59fb service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:33.150870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cee5d2-c604-4bc7-838a-201ad52c59fb service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:33.151148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cee5d2-c604-4bc7-838a-201ad52c59fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:33.159323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cee5d2-c604-4bc7-838a-201ad52c59fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:33.160918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cee5d2-c604-4bc7-838a-201ad52c59fb service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:33.171462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 717/1425] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:34.094870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111020-da75-4b67-b02a-68e69a7040e0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:34.095482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111020-da75-4b67-b02a-68e69a7040e0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:34.095482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111020-da75-4b67-b02a-68e69a7040e0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:34.096809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:34.102048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1bb1-adf3-44e0-9ff7-895d49e47600 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:34.102502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1bb1-adf3-44e0-9ff7-895d49e47600 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:34.102621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1bb1-adf3-44e0-9ff7-895d49e47600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:34.122942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f08a1bb1-adf3-44e0-9ff7-895d49e47600 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:34.124131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f08a1bb1-adf3-44e0-9ff7-895d49e47600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:34.132712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:34.280143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111020-da75-4b67-b02a-68e69a7040e0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:34.303554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111020-da75-4b67-b02a-68e69a7040e0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:34.309564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fc080-52de-4259-8aa3-93da945827bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:34.310204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc080-52de-4259-8aa3-93da945827bb service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:34.310486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc080-52de-4259-8aa3-93da945827bb service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:34.310759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fc080-52de-4259-8aa3-93da945827bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:34.319871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fc080-52de-4259-8aa3-93da945827bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:34.321492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fc080-52de-4259-8aa3-93da945827bb service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:34.331293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 710/1428] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:35.221039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468c19e-d515-4f97-a14e-123b957cab14 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:35.222011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468c19e-d515-4f97-a14e-123b957cab14 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:35.222011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468c19e-d515-4f97-a14e-123b957cab14 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:35.224465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:35.231537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af49c44-8f80-40dc-a580-3ee309d8df92 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:35.232043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af49c44-8f80-40dc-a580-3ee309d8df92 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:35.232144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af49c44-8f80-40dc-a580-3ee309d8df92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:35.253022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9af49c44-8f80-40dc-a580-3ee309d8df92 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:35.254264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9af49c44-8f80-40dc-a580-3ee309d8df92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:35.263956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:35.425120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468c19e-d515-4f97-a14e-123b957cab14 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:35.448568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468c19e-d515-4f97-a14e-123b957cab14 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:35.455107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f72afe-a856-42e5-8c26-81d23837c848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:35.455747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f72afe-a856-42e5-8c26-81d23837c848 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:35.456033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f72afe-a856-42e5-8c26-81d23837c848 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:35.456313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f72afe-a856-42e5-8c26-81d23837c848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:35.464938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f72afe-a856-42e5-8c26-81d23837c848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:35.466573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f72afe-a856-42e5-8c26-81d23837c848 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:35.476121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 720/1431] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:36.348025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d42ed-7e22-4095-8636-2dc914bc8a08 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:36.348603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d42ed-7e22-4095-8636-2dc914bc8a08 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:36.348603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d42ed-7e22-4095-8636-2dc914bc8a08 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:36.350064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:36.356200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0e55c-5327-4a2d-8766-722a73d6918f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:36.356670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0e55c-5327-4a2d-8766-722a73d6918f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:36.356769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0e55c-5327-4a2d-8766-722a73d6918f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:36.381417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-03c0e55c-5327-4a2d-8766-722a73d6918f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:36.382742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-03c0e55c-5327-4a2d-8766-722a73d6918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:36.392151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:36.554478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d42ed-7e22-4095-8636-2dc914bc8a08 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:36.570836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d42ed-7e22-4095-8636-2dc914bc8a08 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:36.577566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b65e0-9e78-4d63-9a0a-377f86effbbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:36.578197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b65e0-9e78-4d63-9a0a-377f86effbbb service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:36.578477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b65e0-9e78-4d63-9a0a-377f86effbbb service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:36.578748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b65e0-9e78-4d63-9a0a-377f86effbbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:36.589203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b65e0-9e78-4d63-9a0a-377f86effbbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:36.590850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b65e0-9e78-4d63-9a0a-377f86effbbb service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:36.600340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 713/1434] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:37.467384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2c0c-fef6-4485-9b8b-4b7c8be4cbaf None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:37.467953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2c0c-fef6-4485-9b8b-4b7c8be4cbaf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:37.467953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c2c0c-fef6-4485-9b8b-4b7c8be4cbaf None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:37.469218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:37.474266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec542e-4131-4e43-9807-d1262a585c21 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:37.474739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec542e-4131-4e43-9807-d1262a585c21 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:37.474960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec542e-4131-4e43-9807-d1262a585c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:37.494094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fdec542e-4131-4e43-9807-d1262a585c21 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:37.495320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fdec542e-4131-4e43-9807-d1262a585c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:37.504721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:37.655594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c2c0c-fef6-4485-9b8b-4b7c8be4cbaf None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:37.675339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c2c0c-fef6-4485-9b8b-4b7c8be4cbaf None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:37.682471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86f42c-6813-44f7-9ef1-3d928bfdf52d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:37.683485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86f42c-6813-44f7-9ef1-3d928bfdf52d service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:37.683937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86f42c-6813-44f7-9ef1-3d928bfdf52d service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:37.684368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86f42c-6813-44f7-9ef1-3d928bfdf52d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:37.695211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86f42c-6813-44f7-9ef1-3d928bfdf52d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:37.696951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86f42c-6813-44f7-9ef1-3d928bfdf52d service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:37.708285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 723/1437] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:38.583202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396c95e-f1fd-4950-adb0-ab2a3daa5847 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:38.583758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396c95e-f1fd-4950-adb0-ab2a3daa5847 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:38.583758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396c95e-f1fd-4950-adb0-ab2a3daa5847 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:38.585343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:38.590603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b65aa-0f61-4e5e-b6ec-2312b1987194 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:38.591073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b65aa-0f61-4e5e-b6ec-2312b1987194 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:38.591158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b65aa-0f61-4e5e-b6ec-2312b1987194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:38.610023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c04b65aa-0f61-4e5e-b6ec-2312b1987194 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:38.611200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c04b65aa-0f61-4e5e-b6ec-2312b1987194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:38.619228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:38.804479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396c95e-f1fd-4950-adb0-ab2a3daa5847 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:38.827141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396c95e-f1fd-4950-adb0-ab2a3daa5847 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:38.833802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c863fd4-c776-48fe-a76b-8a32fa29a6cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:38.834579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c863fd4-c776-48fe-a76b-8a32fa29a6cc service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:38.834863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c863fd4-c776-48fe-a76b-8a32fa29a6cc service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:38.835194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c863fd4-c776-48fe-a76b-8a32fa29a6cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:38.843895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c863fd4-c776-48fe-a76b-8a32fa29a6cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:38.845509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c863fd4-c776-48fe-a76b-8a32fa29a6cc service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:38.854820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 716/1440] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:40.972571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576ab58-8b08-4824-a4d1-fd90f474e3c5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:40.973132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576ab58-8b08-4824-a4d1-fd90f474e3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:40.973132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576ab58-8b08-4824-a4d1-fd90f474e3c5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:40.974474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:40.981616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67934a6-a684-450c-8770-0afc9fd78b44 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:40.982370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67934a6-a684-450c-8770-0afc9fd78b44 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:40.982490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67934a6-a684-450c-8770-0afc9fd78b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:41.004739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f67934a6-a684-450c-8770-0afc9fd78b44 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:41.006381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f67934a6-a684-450c-8770-0afc9fd78b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:41.015954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:41.076131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576ab58-8b08-4824-a4d1-fd90f474e3c5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:41.090811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576ab58-8b08-4824-a4d1-fd90f474e3c5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:41.096450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c43ef-8dcd-4124-9988-b984da0ffbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:41.097610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c43ef-8dcd-4124-9988-b984da0ffbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:41.097993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c43ef-8dcd-4124-9988-b984da0ffbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:41.098440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c43ef-8dcd-4124-9988-b984da0ffbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:41.109916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c43ef-8dcd-4124-9988-b984da0ffbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:41.111408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c43ef-8dcd-4124-9988-b984da0ffbc8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:41.122701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 726/1443] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:43.530865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd7c2e-7eec-4a3c-885a-5e08d321dcfc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:43.531374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd7c2e-7eec-4a3c-885a-5e08d321dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:43.531374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd7c2e-7eec-4a3c-885a-5e08d321dcfc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:43.532727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:43.538358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e36f53-a088-480b-8fcd-ff5f312032cf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:43.538813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e36f53-a088-480b-8fcd-ff5f312032cf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:43.538926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e36f53-a088-480b-8fcd-ff5f312032cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:43.557889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a2e36f53-a088-480b-8fcd-ff5f312032cf None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:43.559082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a2e36f53-a088-480b-8fcd-ff5f312032cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:43.567654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:43.605838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd7c2e-7eec-4a3c-885a-5e08d321dcfc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:43.623506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd7c2e-7eec-4a3c-885a-5e08d321dcfc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:43.629616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375c8b9-57c5-41f1-870e-beb020c85d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:43.630519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375c8b9-57c5-41f1-870e-beb020c85d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:43.630920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375c8b9-57c5-41f1-870e-beb020c85d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:43.631214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375c8b9-57c5-41f1-870e-beb020c85d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:43.640728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375c8b9-57c5-41f1-870e-beb020c85d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:43.642189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375c8b9-57c5-41f1-870e-beb020c85d47 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:43.651385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 719/1446] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:46.239241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec23f8-99f2-4234-9abd-d14445a48670 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:46.239805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec23f8-99f2-4234-9abd-d14445a48670 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:46.239805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec23f8-99f2-4234-9abd-d14445a48670 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:46.241180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:24:46.247803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c2f0-15d3-4ea0-bc8a-678714649a7b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:46.248253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c2f0-15d3-4ea0-bc8a-678714649a7b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:46.248368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c2f0-15d3-4ea0-bc8a-678714649a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:46.275483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c204c2f0-15d3-4ea0-bc8a-678714649a7b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:46.276734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c204c2f0-15d3-4ea0-bc8a-678714649a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:46.286140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:46.324881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec23f8-99f2-4234-9abd-d14445a48670 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:46.337268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec23f8-99f2-4234-9abd-d14445a48670 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:46.341472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39af0f0-edff-416d-8d75-17e90aafaee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:46.342140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39af0f0-edff-416d-8d75-17e90aafaee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:46.342423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39af0f0-edff-416d-8d75-17e90aafaee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:46.342697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39af0f0-edff-416d-8d75-17e90aafaee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:46.353299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39af0f0-edff-416d-8d75-17e90aafaee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:46.354779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39af0f0-edff-416d-8d75-17e90aafaee2 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:46.364229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 729/1449] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:24:48.719869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242c3dc-4a2d-445e-b328-ff8235b6f960 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:48.720521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242c3dc-4a2d-445e-b328-ff8235b6f960 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:48.720521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242c3dc-4a2d-445e-b328-ff8235b6f960 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:48.721819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:48.728544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8fbb0-d34b-40f1-8096-e8eca81d1a9f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:48.729013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8fbb0-d34b-40f1-8096-e8eca81d1a9f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:48.729118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8fbb0-d34b-40f1-8096-e8eca81d1a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:48.757818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c8f8fbb0-d34b-40f1-8096-e8eca81d1a9f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:48.759038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c8f8fbb0-d34b-40f1-8096-e8eca81d1a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:48.768650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:48.811669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242c3dc-4a2d-445e-b328-ff8235b6f960 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:48.826401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242c3dc-4a2d-445e-b328-ff8235b6f960 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:48.830536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bcede-4f32-4f08-bc95-333bab10bf85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:48.831146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bcede-4f32-4f08-bc95-333bab10bf85 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:48.831428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bcede-4f32-4f08-bc95-333bab10bf85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:48.831702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bcede-4f32-4f08-bc95-333bab10bf85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:48.840926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bcede-4f32-4f08-bc95-333bab10bf85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:48.842399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bcede-4f32-4f08-bc95-333bab10bf85 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:48.852273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 722/1452] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:51.107676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696f02-c55e-4938-9d5d-6a4f28c4a3f8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:51.108274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696f02-c55e-4938-9d5d-6a4f28c4a3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:51.108274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696f02-c55e-4938-9d5d-6a4f28c4a3f8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:51.109623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:24:51.115759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d40d-0f7a-4e43-926c-6239b66ce319 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:51.116464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d40d-0f7a-4e43-926c-6239b66ce319 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:51.116620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3d40d-0f7a-4e43-926c-6239b66ce319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:51.140260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d4c3d40d-0f7a-4e43-926c-6239b66ce319 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:51.141467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d4c3d40d-0f7a-4e43-926c-6239b66ce319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:51.150360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:51.188151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696f02-c55e-4938-9d5d-6a4f28c4a3f8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:51.202920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696f02-c55e-4938-9d5d-6a4f28c4a3f8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:51.209368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332d22-eaa6-4d24-ba52-0e4332146820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:51.210134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332d22-eaa6-4d24-ba52-0e4332146820 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:51.210423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332d22-eaa6-4d24-ba52-0e4332146820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:51.210702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332d22-eaa6-4d24-ba52-0e4332146820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:51.220784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332d22-eaa6-4d24-ba52-0e4332146820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:51.222275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332d22-eaa6-4d24-ba52-0e4332146820 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:51.232491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 732/1455] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:53.496150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9394-7ce3-4f48-8171-230ec743c87b None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:53.497101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9394-7ce3-4f48-8171-230ec743c87b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:53.497361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9394-7ce3-4f48-8171-230ec743c87b None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:53.498822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:53.505713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536d19b-6968-4263-8a36-6c25a4315572 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:53.507586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536d19b-6968-4263-8a36-6c25a4315572 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:53.507586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536d19b-6968-4263-8a36-6c25a4315572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:53.527054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2536d19b-6968-4263-8a36-6c25a4315572 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:53.528182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2536d19b-6968-4263-8a36-6c25a4315572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:53.536983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:53.580357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9394-7ce3-4f48-8171-230ec743c87b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:53.593311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9394-7ce3-4f48-8171-230ec743c87b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:53.597322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f17d1c-e0ea-46fb-abf6-94e0fbb2c3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:53.598013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f17d1c-e0ea-46fb-abf6-94e0fbb2c3fd service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:53.598252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f17d1c-e0ea-46fb-abf6-94e0fbb2c3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:53.598549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f17d1c-e0ea-46fb-abf6-94e0fbb2c3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:53.609083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f17d1c-e0ea-46fb-abf6-94e0fbb2c3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:53.610632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f17d1c-e0ea-46fb-abf6-94e0fbb2c3fd service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:53.621008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 725/1458] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:55.985687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f7c90-26e8-4b99-918b-58ad5395a066 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:55.986665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f7c90-26e8-4b99-918b-58ad5395a066 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:55.986665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f7c90-26e8-4b99-918b-58ad5395a066 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:55.988818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:55.995893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504489-c26c-4355-a064-a7eb8ec5e2b2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:55.996330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504489-c26c-4355-a064-a7eb8ec5e2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:55.996443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504489-c26c-4355-a064-a7eb8ec5e2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:56.006605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=70 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:24:56.024435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9b504489-c26c-4355-a064-a7eb8ec5e2b2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:56.025691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9b504489-c26c-4355-a064-a7eb8ec5e2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:56.034139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:56.074587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f7c90-26e8-4b99-918b-58ad5395a066 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:56.084253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=73 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:24:56.091806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f7c90-26e8-4b99-918b-58ad5395a066 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:56.095827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ecb5e-30e3-4b67-b598-52a45035c995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:56.096463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ecb5e-30e3-4b67-b598-52a45035c995 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:56.096743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ecb5e-30e3-4b67-b598-52a45035c995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:56.097012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ecb5e-30e3-4b67-b598-52a45035c995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:56.106520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ecb5e-30e3-4b67-b598-52a45035c995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:56.107953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ecb5e-30e3-4b67-b598-52a45035c995 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:56.117554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 735/1461] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:24:58.349610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafdebd-78ab-48cb-a58c-d2085b65ebd2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:58.350139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafdebd-78ab-48cb-a58c-d2085b65ebd2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:58.350269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafdebd-78ab-48cb-a58c-d2085b65ebd2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:58.351739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:24:58.357629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92983c9-e1b0-4699-9c99-67d3d72595e7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:58.358066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92983c9-e1b0-4699-9c99-67d3d72595e7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:58.358191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92983c9-e1b0-4699-9c99-67d3d72595e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:58.378994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f92983c9-e1b0-4699-9c99-67d3d72595e7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:24:58.380191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f92983c9-e1b0-4699-9c99-67d3d72595e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:24:58.390943 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:24:58.431818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafdebd-78ab-48cb-a58c-d2085b65ebd2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:24:58.449542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafdebd-78ab-48cb-a58c-d2085b65ebd2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:24:58.454725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a7143-1685-4b65-92c9-76805af7b588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:24:58.455370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a7143-1685-4b65-92c9-76805af7b588 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:24:58.455651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a7143-1685-4b65-92c9-76805af7b588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:24:58.455923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a7143-1685-4b65-92c9-76805af7b588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:24:58.466485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a7143-1685-4b65-92c9-76805af7b588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:24:58.467927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a7143-1685-4b65-92c9-76805af7b588 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:24:58.478021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 728/1464] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:25:00.665624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083806fe-b2d0-4315-918d-05967655c759 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:00.666208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083806fe-b2d0-4315-918d-05967655c759 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:00.666208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083806fe-b2d0-4315-918d-05967655c759 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:00.667527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:00.673969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3b6b3-e360-4bf3-b440-dd50e56cf2ee None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:00.674412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3b6b3-e360-4bf3-b440-dd50e56cf2ee None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:00.674536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3b6b3-e360-4bf3-b440-dd50e56cf2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:00.700977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8cc3b6b3-e360-4bf3-b440-dd50e56cf2ee None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:00.702228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8cc3b6b3-e360-4bf3-b440-dd50e56cf2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:00.711522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:00.751641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083806fe-b2d0-4315-918d-05967655c759 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:00.765201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083806fe-b2d0-4315-918d-05967655c759 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:00.769241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447ecec-e84f-42fb-a132-f693625f0727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:00.769898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ecec-e84f-42fb-a132-f693625f0727 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:00.770181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ecec-e84f-42fb-a132-f693625f0727 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:00.770456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ecec-e84f-42fb-a132-f693625f0727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:00.779662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447ecec-e84f-42fb-a132-f693625f0727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:00.781116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447ecec-e84f-42fb-a132-f693625f0727 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:00.791278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 738/1467] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:25:03.015934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f0a06-1f35-4851-9f5f-c128bb7f383e None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:03.016764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f0a06-1f35-4851-9f5f-c128bb7f383e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:03.016912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f0a06-1f35-4851-9f5f-c128bb7f383e None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:03.019075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:25:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:03.025634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b2a62-6034-4763-b7a0-9dcc64784f05 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:03.026089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b2a62-6034-4763-b7a0-9dcc64784f05 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:03.026188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b2a62-6034-4763-b7a0-9dcc64784f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:03.049255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ea2b2a62-6034-4763-b7a0-9dcc64784f05 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:03.050483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea2b2a62-6034-4763-b7a0-9dcc64784f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:03.059852 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:03.096347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f0a06-1f35-4851-9f5f-c128bb7f383e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:03.110911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f0a06-1f35-4851-9f5f-c128bb7f383e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:03.115822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4332c42-84a4-410d-a449-86850aa99a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:03.116489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4332c42-84a4-410d-a449-86850aa99a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:03.116807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4332c42-84a4-410d-a449-86850aa99a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:03.117043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4332c42-84a4-410d-a449-86850aa99a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:03.127356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4332c42-84a4-410d-a449-86850aa99a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:03.128819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4332c42-84a4-410d-a449-86850aa99a38 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:03.138879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 731/1470] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:25:13.043909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=17 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:25:13.134709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=17 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:25:36.209951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b003de-fdfd-4647-af51-aff3ce9146a5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:36.210592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b003de-fdfd-4647-af51-aff3ce9146a5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:36.210592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b003de-fdfd-4647-af51-aff3ce9146a5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:36.211836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 740/1471] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:36.220545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f278e19-3885-427e-9836-d7b69a5fb9fb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:36.220992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f278e19-3885-427e-9836-d7b69a5fb9fb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:36.221102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f278e19-3885-427e-9836-d7b69a5fb9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:36.246724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9f278e19-3885-427e-9836-d7b69a5fb9fb None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:36.247946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f278e19-3885-427e-9836-d7b69a5fb9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:36.257998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 732/1472] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:36.306204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd1e49-7e53-4c62-a694-74b69f066f9d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:36.306578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd1e49-7e53-4c62-a694-74b69f066f9d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:36.306989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd1e49-7e53-4c62-a694-74b69f066f9d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:36.309090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 741/1473] 10.222.0.24 () {54 vars in 962 bytes} [Thu May 19 13:25:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:36.313957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4d13-3e4d-406b-8d44-a7fa013f0606 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:36.314254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4d13-3e4d-406b-8d44-a7fa013f0606 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:36.314544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d4d13-3e4d-406b-8d44-a7fa013f0606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:36.335489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ef8d4d13-3e4d-406b-8d44-a7fa013f0606 None None] MFA Rules not processed for user `7ea7d2d4fd034e66aa589875e3e27ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:36.336746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef8d4d13-3e4d-406b-8d44-a7fa013f0606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:36.344658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 733/1474] 10.222.0.24 () {58 vars in 1059 bytes} [Thu May 19 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:36.352356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd1e49-7e53-4c62-a694-74b69f066f9d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:36.363978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd1e49-7e53-4c62-a694-74b69f066f9d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:36.368226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fde4f-eca6-42a3-9c69-0272bf81f975 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:36.368878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fde4f-eca6-42a3-9c69-0272bf81f975 service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:36.369163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fde4f-eca6-42a3-9c69-0272bf81f975 service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:36.369442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fde4f-eca6-42a3-9c69-0272bf81f975 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:36.379552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fde4f-eca6-42a3-9c69-0272bf81f975 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:36.381160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fde4f-eca6-42a3-9c69-0272bf81f975 service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:36.390681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 742/1475] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:25:38.700734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303b114-f851-4606-af2d-f5631f495584 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:38.701657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303b114-f851-4606-af2d-f5631f495584 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:38.701813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303b114-f851-4606-af2d-f5631f495584 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:38.703863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 734/1476] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:38.709735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e15ae-3414-4515-9365-1032bee15f33 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:38.710171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e15ae-3414-4515-9365-1032bee15f33 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:38.710283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e15ae-3414-4515-9365-1032bee15f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:38.730493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b44e15ae-3414-4515-9365-1032bee15f33 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:38.731702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b44e15ae-3414-4515-9365-1032bee15f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:38.745442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 743/1477] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:38.763792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303b114-f851-4606-af2d-f5631f495584 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:38.776714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303b114-f851-4606-af2d-f5631f495584 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:38.781185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:38.781874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:38.782160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:38.782438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:38.783177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:38.784966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:25:38.784966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:25:38.784966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:25:38.786399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf1e8f-0563-4cc1-bd2f-7c6aa20ec29e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:38.807468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 735/1478] 10.222.0.24 () {56 vars in 1108 bytes} [Thu May 19 13:25:38 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:25:41.203501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60ae77-a31a-4e96-9d3e-b03dd8f6f249 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:41.204110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60ae77-a31a-4e96-9d3e-b03dd8f6f249 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:41.204590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60ae77-a31a-4e96-9d3e-b03dd8f6f249 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:41.208128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 744/1479] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:41.213831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf4595-d8ce-4e61-a7fb-2151f9b9c266 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:41.214330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf4595-d8ce-4e61-a7fb-2151f9b9c266 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:41.214330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf4595-d8ce-4e61-a7fb-2151f9b9c266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:41.235923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9fdf4595-d8ce-4e61-a7fb-2151f9b9c266 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:41.237119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9fdf4595-d8ce-4e61-a7fb-2151f9b9c266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:41.248177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 736/1480] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:41.295636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60ae77-a31a-4e96-9d3e-b03dd8f6f249 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:41.312992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60ae77-a31a-4e96-9d3e-b03dd8f6f249 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:41.319354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5270ae-ceb6-4f99-82cd-fcbf2b5c1fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:41.319994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5270ae-ceb6-4f99-82cd-fcbf2b5c1fb4 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:41.320394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5270ae-ceb6-4f99-82cd-fcbf2b5c1fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:41.320670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5270ae-ceb6-4f99-82cd-fcbf2b5c1fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:41.329759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5270ae-ceb6-4f99-82cd-fcbf2b5c1fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:41.331452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5270ae-ceb6-4f99-82cd-fcbf2b5c1fb4 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:41.344466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 745/1481] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:25:43.760247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bde6d1-37e3-46ed-b5a1-81f208d31108 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:43.761092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bde6d1-37e3-46ed-b5a1-81f208d31108 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:43.761092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bde6d1-37e3-46ed-b5a1-81f208d31108 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:43.763294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 737/1482] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:43.771662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a49b9-2752-419b-8eb7-347fcb1a96a8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:43.772099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a49b9-2752-419b-8eb7-347fcb1a96a8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:43.772205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a49b9-2752-419b-8eb7-347fcb1a96a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:43.793161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a62a49b9-2752-419b-8eb7-347fcb1a96a8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:43.794384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a62a49b9-2752-419b-8eb7-347fcb1a96a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:43.806180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 746/1483] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:43.850487 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bde6d1-37e3-46ed-b5a1-81f208d31108 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:43.862854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bde6d1-37e3-46ed-b5a1-81f208d31108 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:43.867288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823c19-2b3a-4ff3-91c5-d7d997eed03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:43.867999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823c19-2b3a-4ff3-91c5-d7d997eed03c service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:43.868283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823c19-2b3a-4ff3-91c5-d7d997eed03c service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:43.868574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823c19-2b3a-4ff3-91c5-d7d997eed03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:43.878681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823c19-2b3a-4ff3-91c5-d7d997eed03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:43.880695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823c19-2b3a-4ff3-91c5-d7d997eed03c service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:43.890618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 738/1484] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:25:46.164338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fd909-941c-4cd4-b0d7-d5e18b10ad42 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:46.165686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fd909-941c-4cd4-b0d7-d5e18b10ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:46.165686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fd909-941c-4cd4-b0d7-d5e18b10ad42 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:46.167897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 747/1485] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:46.174278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da8c0c-58fe-4938-b830-5eeede45fb54 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:46.175003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da8c0c-58fe-4938-b830-5eeede45fb54 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:46.175179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da8c0c-58fe-4938-b830-5eeede45fb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:46.194970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10da8c0c-58fe-4938-b830-5eeede45fb54 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:46.196188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10da8c0c-58fe-4938-b830-5eeede45fb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:46.204881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 739/1486] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:46.242351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fd909-941c-4cd4-b0d7-d5e18b10ad42 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:46.253873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fd909-941c-4cd4-b0d7-d5e18b10ad42 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:46.257670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8441f5-4e9a-4f1e-8d0d-3502befa3b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:46.258318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8441f5-4e9a-4f1e-8d0d-3502befa3b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:46.258601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8441f5-4e9a-4f1e-8d0d-3502befa3b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:46.258876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8441f5-4e9a-4f1e-8d0d-3502befa3b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:46.269447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8441f5-4e9a-4f1e-8d0d-3502befa3b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:46.271233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8441f5-4e9a-4f1e-8d0d-3502befa3b7d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:46.280886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 748/1487] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:25:48.605721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44597a7d-93e3-4eda-9199-1344032f481d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:48.606270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44597a7d-93e3-4eda-9199-1344032f481d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:48.606270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44597a7d-93e3-4eda-9199-1344032f481d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:48.607622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 740/1488] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:48.613264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face7499-8d14-495c-ab49-8aee0f105389 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:48.613750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face7499-8d14-495c-ab49-8aee0f105389 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:48.613927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face7499-8d14-495c-ab49-8aee0f105389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:48.632642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-face7499-8d14-495c-ab49-8aee0f105389 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:48.633889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-face7499-8d14-495c-ab49-8aee0f105389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:48.643213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 749/1489] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:48.687658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44597a7d-93e3-4eda-9199-1344032f481d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:48.700292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44597a7d-93e3-4eda-9199-1344032f481d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:48.704133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee0cc6-f64b-4c75-ab5e-d6b93b50e9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:48.704768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee0cc6-f64b-4c75-ab5e-d6b93b50e9bf service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:48.705054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee0cc6-f64b-4c75-ab5e-d6b93b50e9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:48.705353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee0cc6-f64b-4c75-ab5e-d6b93b50e9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:48.715917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee0cc6-f64b-4c75-ab5e-d6b93b50e9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:48.717386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee0cc6-f64b-4c75-ab5e-d6b93b50e9bf service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:48.727568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 741/1490] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:25:51.185254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577676b-bd8a-4603-989d-8e86a20d5d83 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:51.185916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577676b-bd8a-4603-989d-8e86a20d5d83 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:51.185916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577676b-bd8a-4603-989d-8e86a20d5d83 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:51.187207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 750/1491] 10.222.0.24 () {54 vars in 926 bytes} [Thu May 19 13:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:25:51.193963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757015f-901f-4ed3-aa7a-fd0a37b29062 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:51.194618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757015f-901f-4ed3-aa7a-fd0a37b29062 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:51.194845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757015f-901f-4ed3-aa7a-fd0a37b29062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:51.218654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7757015f-901f-4ed3-aa7a-fd0a37b29062 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:25:51.219901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7757015f-901f-4ed3-aa7a-fd0a37b29062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:25:51.229360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 742/1492] 10.222.0.24 () {58 vars in 1023 bytes} [Thu May 19 13:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:25:51.267804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577676b-bd8a-4603-989d-8e86a20d5d83 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:25:51.279769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577676b-bd8a-4603-989d-8e86a20d5d83 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:25:51.283675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adb372-2eab-472c-9c31-ce207b843849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:25:51.284306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb372-2eab-472c-9c31-ce207b843849 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:25:51.284585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb372-2eab-472c-9c31-ce207b843849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:25:51.284855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adb372-2eab-472c-9c31-ce207b843849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:25:51.295386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adb372-2eab-472c-9c31-ce207b843849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:25:51.296826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adb372-2eab-472c-9c31-ce207b843849 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:25:51.306727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 751/1493] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:26:01.213119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=33 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:26:01.302232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=33 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:27:31.820989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:31.821530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:31.821530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:31.842609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:27:31.843786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:31.852436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 743/1494] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:27:31.862831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:31.863378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:31.863378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:31.891968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:27:31.893314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:31.901940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 752/1495] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:27:31.906801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:31.916998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41791ef6-444f-4572-9a81-3121c25a034a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:31.921696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:31.922356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:31.922639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:31.922930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:31.924137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:27:31.924137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:31.925286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:31.925286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:27:31.925286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:27:31.926230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:31.938098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 744/1496] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:27:31.946429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:31.956419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b43b1-d3fd-4421-9a10-cdab705debe7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:31.962313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:31.962974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:31.963256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:31.963530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:31.964230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:31.966107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:31.966107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:27:31.966107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:27:31.967321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.015859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 753/1497] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:27:31 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:27:32.020731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.031247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84eac2-e934-4c77-89a1-dd2f626cd0cb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.090868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.091457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.091731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.091997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.092682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.093724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:32.093724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:27:32.093724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:27:32.094936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.115837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 745/1498] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:27:32 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:27:32.120048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.131107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca1b53-eed9-4530-ad28-71a03cb60d05 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.135975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.136639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.136923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.137201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.138052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.139091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:32.139091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:27:32.139091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:27:32.139970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.152930 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 754/1499] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:27:32.159104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.169847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154ee3c-49ef-4d51-aa04-340e22d47184 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.175829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.176500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.176785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.177058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.177819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.178845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:32.178845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:27:32.178845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:27:32.179731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3c224-3f62-46ae-a483-08be6e0afd55 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.194186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 746/1500] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:27:32.199218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.215296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726300d3-e650-401a-8be4-78bf7d4da1da admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.222120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.223004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.223290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.223570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] PATH_INFO: `/v3/projects/5e47f6c21a4249018b2ffdcfa26c57bd/users/485980781c38492e88c241bc822380b1/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.238741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e47f6c21a4249018b2ffdcfa26c57bd, user_id=485980781c38492e88c241bc822380b1, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.239790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:32.239790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:27:32.239790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:27:32.241792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7988843-927b-4a88-ab1e-9d9329475db7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.257288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 755/1501] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:27:32 2022] PUT /identity/v3/projects/5e47f6c21a4249018b2ffdcfa26c57bd/users/485980781c38492e88c241bc822380b1/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:27:32.268900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a64d93-f885-4ad9-824d-887b3fd5e95e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.269194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a64d93-f885-4ad9-824d-887b3fd5e95e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.269512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a64d93-f885-4ad9-824d-887b3fd5e95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.297373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-85a64d93-f885-4ad9-824d-887b3fd5e95e None None] MFA Rules not processed for user `485980781c38492e88c241bc822380b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:27:32.298687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-85a64d93-f885-4ad9-824d-887b3fd5e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:32.367081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 747/1502] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:27:32.381232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fba88-adbf-4863-9fa1-547982ef398f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.381544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fba88-adbf-4863-9fa1-547982ef398f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.381826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fba88-adbf-4863-9fa1-547982ef398f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.383185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 756/1503] 10.222.0.24 () {54 vars in 963 bytes} [Thu May 19 13:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:27:32.388236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb420773-e956-4db5-88cb-46c145d16ecb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.388531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb420773-e956-4db5-88cb-46c145d16ecb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.388813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb420773-e956-4db5-88cb-46c145d16ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.409238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fb420773-e956-4db5-88cb-46c145d16ecb None None] MFA Rules not processed for user `d92108b779ca47aca8f941b7dfae80d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:27:32.410534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fb420773-e956-4db5-88cb-46c145d16ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:32.466883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:27:32.476300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 748/1504] 10.222.0.24 () {58 vars in 1060 bytes} [Thu May 19 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:27:32.488474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fba88-adbf-4863-9fa1-547982ef398f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.500324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fba88-adbf-4863-9fa1-547982ef398f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.504624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.505402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.505712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.505989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.515041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.516484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.525542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 757/1505] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:27:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:27:32.569011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fe193-9679-4ec2-841c-cea63b9d2ca2 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.569385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fe193-9679-4ec2-841c-cea63b9d2ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.569846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fe193-9679-4ec2-841c-cea63b9d2ca2 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.571864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 749/1506] 10.222.0.24 () {52 vars in 829 bytes} [Thu May 19 13:27:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:27:32.575485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.644474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917a8e7-6a2c-48b2-a36b-33491faddc00 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.649918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.650837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.651554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.651554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] PATH_INFO: `/v3/users/485980781c38492e88c241bc822380b1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.652676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=485980781c38492e88c241bc822380b1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.653728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:32.653728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:27:32.653728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:27:32.654941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.723016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485980781c38492e88c241bc822380b1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:32.743280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485980781c38492e88c241bc822380b1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:32.744033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485980781c38492e88c241bc822380b1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:32.744926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485980781c38492e88c241bc822380b1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:32.758035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485980781c38492e88c241bc822380b1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:32.762217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:27:32.775602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 758/1507] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:27:32 2022] DELETE /identity/v3/users/485980781c38492e88c241bc822380b1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:27:32.781721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b3991-4736-482c-9ade-961b733771e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.782018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b3991-4736-482c-9ade-961b733771e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.782297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b3991-4736-482c-9ade-961b733771e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.802360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-933b3991-4736-482c-9ade-961b733771e4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:27:32.803583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-933b3991-4736-482c-9ade-961b733771e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:32.878395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 750/1508] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:27:32.889189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:32.891051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:32.952584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746d371-769f-4be5-9760-38e9bb8ef44c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:32.956559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b8d8-e201-402f-9570-1b351f5ed0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:32.957217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b8d8-e201-402f-9570-1b351f5ed0ef service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:32.957542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b8d8-e201-402f-9570-1b351f5ed0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:32.957815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b8d8-e201-402f-9570-1b351f5ed0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:32.967226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947b8d8-e201-402f-9570-1b351f5ed0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:32.968685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947b8d8-e201-402f-9570-1b351f5ed0ef service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:32.978824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 759/1509] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:27:33.396283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b3991-4736-482c-9ade-961b733771e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:27:33.397842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-933b3991-4736-482c-9ade-961b733771e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:27:33.412286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b3991-4736-482c-9ade-961b733771e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:27:33.418413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:27:33.419561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:27:33.419894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:27:33.420176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] PATH_INFO: `/v3/projects/5e47f6c21a4249018b2ffdcfa26c57bd` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:27:33.421483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e47f6c21a4249018b2ffdcfa26c57bd)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:27:33.422521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:27:33.422521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:27:33.422521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:27:33.423832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:27:33.488693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e47f6c21a4249018b2ffdcfa26c57bd', 'request_id': 'req-207dd8ca-9427-4818-9943-5a6d1ddb972e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:33.503744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e47f6c21a4249018b2ffdcfa26c57bd', 'request_id': 'req-207dd8ca-9427-4818-9943-5a6d1ddb972e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:33.510781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e47f6c21a4249018b2ffdcfa26c57bd', 'request_id': 'req-207dd8ca-9427-4818-9943-5a6d1ddb972e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:33.531276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] The token cache is being invalidate because project 5e47f6c21a4249018b2ffdcfa26c57bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:27:33.531524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:33.532338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-207dd8ca-9427-4818-9943-5a6d1ddb972e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:27:33.534275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 751/1510] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:27:33 2022] DELETE /identity/v3/projects/5e47f6c21a4249018b2ffdcfa26c57bd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:27:42.974483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:27:43.527882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:29:16.609021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5835e4-c58d-43dd-8b63-7c6c941d9808 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:16.609996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5835e4-c58d-43dd-8b63-7c6c941d9808 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:16.609996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5835e4-c58d-43dd-8b63-7c6c941d9808 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:16.612106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:29:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:16.619358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eab176-8757-42fc-afc7-3a1956133246 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:16.619825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eab176-8757-42fc-afc7-3a1956133246 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:16.619931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eab176-8757-42fc-afc7-3a1956133246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:16.639407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-02eab176-8757-42fc-afc7-3a1956133246 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:16.640642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02eab176-8757-42fc-afc7-3a1956133246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:16.703958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:16.883199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5835e4-c58d-43dd-8b63-7c6c941d9808 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:16.885378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c5835e4-c58d-43dd-8b63-7c6c941d9808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:16.967118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5835e4-c58d-43dd-8b63-7c6c941d9808 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:16.973441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a86dae-816d-447f-918f-b55fed193bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:16.974117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a86dae-816d-447f-918f-b55fed193bb4 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:16.974403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a86dae-816d-447f-918f-b55fed193bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:16.974680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a86dae-816d-447f-918f-b55fed193bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:16.983405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a86dae-816d-447f-918f-b55fed193bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:16.985024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a86dae-816d-447f-918f-b55fed193bb4 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:16.995402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 761/1513] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:17.128322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db48df3-ccbc-4c3c-afa1-7a77d42dc88f None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:17.128531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db48df3-ccbc-4c3c-afa1-7a77d42dc88f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:17.128949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db48df3-ccbc-4c3c-afa1-7a77d42dc88f None None] PATH_INFO: `/v3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:17.130531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 753/1514] 10.222.0.58 () {54 vars in 925 bytes} [Thu May 19 13:29:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:29:17.138219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7acba-069b-4832-8354-0540391f1804 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:17.138521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7acba-069b-4832-8354-0540391f1804 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:17.138797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d7acba-069b-4832-8354-0540391f1804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:17.158481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-32d7acba-069b-4832-8354-0540391f1804 None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:17.159702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-32d7acba-069b-4832-8354-0540391f1804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:17.224500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 762/1515] 10.222.0.58 () {58 vars in 1015 bytes} [Thu May 19 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:17.927472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95faaeb-3ff6-4214-aad5-819e791554b4 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:17.928159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95faaeb-3ff6-4214-aad5-819e791554b4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:17.928159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95faaeb-3ff6-4214-aad5-819e791554b4 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:17.929434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 754/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:17.934425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c149e6-d609-40fb-8652-34fb6725220c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:17.934846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c149e6-d609-40fb-8652-34fb6725220c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:17.934980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c149e6-d609-40fb-8652-34fb6725220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:17.953501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-21c149e6-d609-40fb-8652-34fb6725220c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:17.954689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21c149e6-d609-40fb-8652-34fb6725220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:17.963501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 763/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:18.121514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95faaeb-3ff6-4214-aad5-819e791554b4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:18.138797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95faaeb-3ff6-4214-aad5-819e791554b4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:18.145471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8468-1743-4e50-ade4-96f4b0fc0160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:18.146133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8468-1743-4e50-ade4-96f4b0fc0160 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:18.146422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8468-1743-4e50-ade4-96f4b0fc0160 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:18.146700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8468-1743-4e50-ade4-96f4b0fc0160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:18.155398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e8468-1743-4e50-ade4-96f4b0fc0160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:18.157014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e8468-1743-4e50-ade4-96f4b0fc0160 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:18.172911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 755/1518] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:26.175995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1278738-351b-46a5-8987-c4dcd4fca403 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:26.176938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1278738-351b-46a5-8987-c4dcd4fca403 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:26.176938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1278738-351b-46a5-8987-c4dcd4fca403 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:26.178973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 764/1519] 10.222.0.24 () {54 vars in 960 bytes} [Thu May 19 13:29:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:26.187181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8264806-c253-47dd-b2ef-8d121d55f198 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:26.187644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8264806-c253-47dd-b2ef-8d121d55f198 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:26.187746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8264806-c253-47dd-b2ef-8d121d55f198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:26.209763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c8264806-c253-47dd-b2ef-8d121d55f198 None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:26.211004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c8264806-c253-47dd-b2ef-8d121d55f198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:26.220798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 756/1520] 10.222.0.24 () {58 vars in 1057 bytes} [Thu May 19 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:26.228555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1278738-351b-46a5-8987-c4dcd4fca403 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:26.242920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1278738-351b-46a5-8987-c4dcd4fca403 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:26.248787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e45560-9cc9-464c-a45b-80e766562958 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:26.249458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e45560-9cc9-464c-a45b-80e766562958 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:26.249744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e45560-9cc9-464c-a45b-80e766562958 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:26.250018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e45560-9cc9-464c-a45b-80e766562958 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:26.259359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e45560-9cc9-464c-a45b-80e766562958 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:26.260974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e45560-9cc9-464c-a45b-80e766562958 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:26.271489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 765/1521] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:29.523443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9699-382f-40b3-8480-a6d2301fc6af None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:29.524004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9699-382f-40b3-8480-a6d2301fc6af None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:29.524004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9699-382f-40b3-8480-a6d2301fc6af None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:29.525297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:29.531358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f906a4-2566-4012-87d4-a2aa212d0841 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:29.532134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f906a4-2566-4012-87d4-a2aa212d0841 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:29.532312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f906a4-2566-4012-87d4-a2aa212d0841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:29.559622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-37f906a4-2566-4012-87d4-a2aa212d0841 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:29.560838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-37f906a4-2566-4012-87d4-a2aa212d0841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:29.570288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:29.734870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9699-382f-40b3-8480-a6d2301fc6af None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:29.752595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9699-382f-40b3-8480-a6d2301fc6af None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:29.758452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f6db2-066f-482a-ac85-2829ef95748f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:29.759092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f6db2-066f-482a-ac85-2829ef95748f service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:29.759375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f6db2-066f-482a-ac85-2829ef95748f service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:29.759650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f6db2-066f-482a-ac85-2829ef95748f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:29.767985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f6db2-066f-482a-ac85-2829ef95748f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:29.769615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f6db2-066f-482a-ac85-2829ef95748f service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:29.779397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 758/1524] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:30.662647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0a3cf-2f5a-4e6a-8df7-5439e6055abc None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:30.663483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0a3cf-2f5a-4e6a-8df7-5439e6055abc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:30.663483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0a3cf-2f5a-4e6a-8df7-5439e6055abc None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:30.666135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 13:29:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:30.674318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2ee13-57ef-40ae-ae1f-d05ac6f4a393 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:30.674749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2ee13-57ef-40ae-ae1f-d05ac6f4a393 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:30.674896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2ee13-57ef-40ae-ae1f-d05ac6f4a393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:30.701849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a5b2ee13-57ef-40ae-ae1f-d05ac6f4a393 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:30.703055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a5b2ee13-57ef-40ae-ae1f-d05ac6f4a393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:30.711994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:30.865739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0a3cf-2f5a-4e6a-8df7-5439e6055abc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:30.887857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0a3cf-2f5a-4e6a-8df7-5439e6055abc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:30.894360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e6469-ed05-4feb-847d-40a40bdb8888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:30.895207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e6469-ed05-4feb-847d-40a40bdb8888 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:30.895846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e6469-ed05-4feb-847d-40a40bdb8888 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:30.896130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e6469-ed05-4feb-847d-40a40bdb8888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:30.905252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e6469-ed05-4feb-847d-40a40bdb8888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:30.906890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e6469-ed05-4feb-847d-40a40bdb8888 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:30.917316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 768/1527] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:35.030362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170b654-5dc7-45f3-94ff-3433a812b537 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:35.030932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170b654-5dc7-45f3-94ff-3433a812b537 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:35.030932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170b654-5dc7-45f3-94ff-3433a812b537 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:35.032312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:29:35.039003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830c90a-33b2-49fd-ba21-cd942e720b69 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:35.039474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830c90a-33b2-49fd-ba21-cd942e720b69 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:35.039573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830c90a-33b2-49fd-ba21-cd942e720b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:35.069380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7830c90a-33b2-49fd-ba21-cd942e720b69 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:35.070626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7830c90a-33b2-49fd-ba21-cd942e720b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:35.083152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:35.119565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170b654-5dc7-45f3-94ff-3433a812b537 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:35.121605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f170b654-5dc7-45f3-94ff-3433a812b537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:35.187379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170b654-5dc7-45f3-94ff-3433a812b537 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:35.192381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10dde1-9771-43f0-8ddb-3e923efac1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:35.193148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10dde1-9771-43f0-8ddb-3e923efac1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:35.193435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10dde1-9771-43f0-8ddb-3e923efac1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:35.193749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10dde1-9771-43f0-8ddb-3e923efac1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:35.204669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10dde1-9771-43f0-8ddb-3e923efac1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:35.206190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10dde1-9771-43f0-8ddb-3e923efac1f0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:35.216177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 761/1530] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:37.462827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21a44-7b60-4889-9018-17a88b762e18 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:37.463400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21a44-7b60-4889-9018-17a88b762e18 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:37.463400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a21a44-7b60-4889-9018-17a88b762e18 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:37.465030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:37.470714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace02a3-cc0a-4685-ba4b-e708e9d9cb86 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:37.471192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace02a3-cc0a-4685-ba4b-e708e9d9cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:37.471287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace02a3-cc0a-4685-ba4b-e708e9d9cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:37.490761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-aace02a3-cc0a-4685-ba4b-e708e9d9cb86 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:37.491971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aace02a3-cc0a-4685-ba4b-e708e9d9cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:37.500363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:29:37.538822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21a44-7b60-4889-9018-17a88b762e18 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:37.558109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a21a44-7b60-4889-9018-17a88b762e18 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:37.563701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78829d7-2f8c-4ac2-b7e0-e8420797be23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:37.564340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78829d7-2f8c-4ac2-b7e0-e8420797be23 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:37.564623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78829d7-2f8c-4ac2-b7e0-e8420797be23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:37.564900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78829d7-2f8c-4ac2-b7e0-e8420797be23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:37.576120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78829d7-2f8c-4ac2-b7e0-e8420797be23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:37.578429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78829d7-2f8c-4ac2-b7e0-e8420797be23 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:37.594356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 771/1533] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:41.620913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395513e5-f269-4982-8e00-32c6543f14cf None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:41.621466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395513e5-f269-4982-8e00-32c6543f14cf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:41.621466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395513e5-f269-4982-8e00-32c6543f14cf None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:41.622910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:41.630034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ae119-dec8-4538-9615-6700db6f8fe9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:41.630531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ae119-dec8-4538-9615-6700db6f8fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:41.631017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ae119-dec8-4538-9615-6700db6f8fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:41.654343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-440ae119-dec8-4538-9615-6700db6f8fe9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:41.655542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-440ae119-dec8-4538-9615-6700db6f8fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:41.663527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:41.959537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395513e5-f269-4982-8e00-32c6543f14cf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:41.962517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-395513e5-f269-4982-8e00-32c6543f14cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:42.043573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395513e5-f269-4982-8e00-32c6543f14cf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:42.048200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c161e543-d2c0-4cf1-a5f7-46359519ea95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:42.049313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e543-d2c0-4cf1-a5f7-46359519ea95 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:42.049867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e543-d2c0-4cf1-a5f7-46359519ea95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:42.050362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e543-d2c0-4cf1-a5f7-46359519ea95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:42.065678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161e543-d2c0-4cf1-a5f7-46359519ea95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:42.070761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c161e543-d2c0-4cf1-a5f7-46359519ea95 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:42.077316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:29:42.086708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 764/1536] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:44.421277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcd0d2-be01-48aa-8015-aaa5b9cbb7fc None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:44.422291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcd0d2-be01-48aa-8015-aaa5b9cbb7fc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:44.422291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcd0d2-be01-48aa-8015-aaa5b9cbb7fc None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:44.424521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 13:29:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:29:44.430346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23c48-2ed9-41e3-bd3f-54c46d619ea6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:44.430818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23c48-2ed9-41e3-bd3f-54c46d619ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:44.430924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23c48-2ed9-41e3-bd3f-54c46d619ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:44.449956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-88a23c48-2ed9-41e3-bd3f-54c46d619ea6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:29:44.451161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-88a23c48-2ed9-41e3-bd3f-54c46d619ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:44.459924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:29:44.729078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcd0d2-be01-48aa-8015-aaa5b9cbb7fc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:29:44.731335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d2bcd0d2-be01-48aa-8015-aaa5b9cbb7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:29:44.740040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:29:44.746993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcd0d2-be01-48aa-8015-aaa5b9cbb7fc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:29:44.751175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a70a3-3447-4c24-81e1-35692abf0545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:29:44.751820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a70a3-3447-4c24-81e1-35692abf0545 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:29:44.752105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a70a3-3447-4c24-81e1-35692abf0545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:29:44.752340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a70a3-3447-4c24-81e1-35692abf0545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:29:44.762032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a70a3-3447-4c24-81e1-35692abf0545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:29:44.763534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a70a3-3447-4c24-81e1-35692abf0545 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:29:44.773156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 774/1539] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:29:54.444662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:29:54.769143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:15.447397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.448062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.448062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.481643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.482918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.493944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 766/1540] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:15.505603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9259e0-7041-491e-a212-fc28061644d6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.506652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9259e0-7041-491e-a212-fc28061644d6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.507150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9259e0-7041-491e-a212-fc28061644d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.535216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2c9259e0-7041-491e-a212-fc28061644d6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.536393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2c9259e0-7041-491e-a212-fc28061644d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.545791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 775/1541] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.550563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.553383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034568-116c-40fe-b5b7-3fd0fe6bd53c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.553704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034568-116c-40fe-b5b7-3fd0fe6bd53c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.554416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034568-116c-40fe-b5b7-3fd0fe6bd53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.565153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbe1ee-cb26-4025-ad8b-126aba997032 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.569991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.570683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.570968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.571401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.572146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:15.572456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:15.574122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f10008-33e2-4f64-a96e-68e911e9ca22 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:15.586070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-84034568-116c-40fe-b5b7-3fd0fe6bd53c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.587286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-84034568-116c-40fe-b5b7-3fd0fe6bd53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.595322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 767/1542] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:15.596615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.596910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.597193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.604387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 776/1543] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:15.605920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab74421-a752-4307-b0c4-1d845b488e4b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.606213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab74421-a752-4307-b0c4-1d845b488e4b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.606492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab74421-a752-4307-b0c4-1d845b488e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.629964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.631218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.648843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 768/1544] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.648843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.649148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1ab74421-a752-4307-b0c4-1d845b488e4b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.650245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1ab74421-a752-4307-b0c4-1d845b488e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.669853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 777/1545] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.671036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd8a5-ad6c-48a0-a26b-8ce9b6ec84a3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.671326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd8a5-ad6c-48a0-a26b-8ce9b6ec84a3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.671608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddd8a5-ad6c-48a0-a26b-8ce9b6ec84a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.679941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701538a6-9845-4ef8-b1d0-822daf8d766f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.684634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.685315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.685628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.685944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.686658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:15.687842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:31:15.687842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:31:15.687842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:31:15.689072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fca900-6913-49a4-a71f-eeffbf7f558f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:15.707409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-73ddd8a5-ad6c-48a0-a26b-8ce9b6ec84a3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.709001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73ddd8a5-ad6c-48a0-a26b-8ce9b6ec84a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.719040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 778/1546] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.721139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d624a-2055-4db6-b1a8-cf79c5dcb736 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.721683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d624a-2055-4db6-b1a8-cf79c5dcb736 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.722230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d624a-2055-4db6-b1a8-cf79c5dcb736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.727895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 769/1547] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:15.729340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd169-1a97-4ec5-9801-465c8672a39e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.729654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd169-1a97-4ec5-9801-465c8672a39e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.729933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd169-1a97-4ec5-9801-465c8672a39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.744236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-879d624a-2055-4db6-b1a8-cf79c5dcb736 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.745974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-879d624a-2055-4db6-b1a8-cf79c5dcb736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.750793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-96dfd169-1a97-4ec5-9801-465c8672a39e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.752320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-96dfd169-1a97-4ec5-9801-465c8672a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.813354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 779/1548] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.814951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.815243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.815523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.826750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 770/1549] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.828022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.828311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.828970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.834909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.836117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.844639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 780/1550] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.845437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.846222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.847396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.855517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 771/1551] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.856210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.858609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332368b4-2aa0-4914-a6df-85143d5aef7e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.863119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.863782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.864067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.864420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.865148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:15.865482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:15.865773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0a96-88ab-4135-9d53-7d62a6ae1854 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.866510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 13:31:15.866510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:31:15.866510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:31:15.867398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3893f-5758-45eb-860c-9b3f13e1b424 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:15.870498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8926147-4bc0-4688-b392-0e19d7b4c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.871153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8926147-4bc0-4688-b392-0e19d7b4c851 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.871432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8926147-4bc0-4688-b392-0e19d7b4c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.871699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8926147-4bc0-4688-b392-0e19d7b4c851 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.872387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8926147-4bc0-4688-b392-0e19d7b4c851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:15.874474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8926147-4bc0-4688-b392-0e19d7b4c851 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:15.879452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 781/1552] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:15.880717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.881091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.881294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.896256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 772/1553] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:15.897559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.897848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.898115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.901027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.902276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.910765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 782/1554] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.911997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.917654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.918867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.925368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cde857-57f0-4d5d-80d5-14539daa1965 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.927402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 773/1555] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.928255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.930287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.930993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.931275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.931548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.932391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:15.932890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:15.935212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e011cd-630f-4442-8f1c-36b86241d837 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:15.937773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce5e5-025a-4c36-889e-cdba49ab72b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.942319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.942966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.943250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.943524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.944244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:15.944548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:15.946175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d0832-1845-4ddf-9618-cc8d9676aaa2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:15.949437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 783/1556] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:15.950143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069c462-6abd-4c08-98c4-71273811178d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.950441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069c462-6abd-4c08-98c4-71273811178d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.950721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069c462-6abd-4c08-98c4-71273811178d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.965332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 774/1557] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:15.966666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.966954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.967337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.973632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f069c462-6abd-4c08-98c4-71273811178d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.974903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f069c462-6abd-4c08-98c4-71273811178d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.982889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 784/1558] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.983747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069c462-6abd-4c08-98c4-71273811178d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.988008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:15.989573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:15.993256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069c462-6abd-4c08-98c4-71273811178d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:15.997593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 775/1559] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:15.997799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14b1e8-0699-41ff-a938-35235ae40a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:15.998223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:15.998700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14b1e8-0699-41ff-a938-35235ae40a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:15.998989 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14b1e8-0699-41ff-a938-35235ae40a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:15.999257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14b1e8-0699-41ff-a938-35235ae40a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:15.999947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14b1e8-0699-41ff-a938-35235ae40a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.002022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14b1e8-0699-41ff-a938-35235ae40a56 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.007774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a536fbd-c26a-4aa2-afe3-457da52e86c0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.012349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae994b-8f6e-4a28-95dc-c5750a570d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.012995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae994b-8f6e-4a28-95dc-c5750a570d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.013277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae994b-8f6e-4a28-95dc-c5750a570d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.013594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae994b-8f6e-4a28-95dc-c5750a570d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.014377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae994b-8f6e-4a28-95dc-c5750a570d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.015961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae994b-8f6e-4a28-95dc-c5750a570d59 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.026122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 776/1560] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.027335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.031084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.031211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 785/1561] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:15 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.031330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.031442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.031442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.031442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.047423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:16.047678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:16.048618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:16.048821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:16.091523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:16.107992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 777/1562] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:16.112145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.112567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 786/1563] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:16.112567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.121168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ffc96-8b1d-4892-9bce-0470387ec1ba None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.121371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156796a-cb99-49ed-8c7e-a73572ce7f88 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.126523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.126709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.127169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.127448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.127705 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.127873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.128181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.128400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.128569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.129327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.130505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.131263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.133840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:16.157649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 778/1564] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.159172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 787/1565] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.161795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.161950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.222882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81038456-66c9-459c-a79c-bb1549eb8445 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.227440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806b660-b02a-4f42-8622-aefc58788402 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.227669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.228326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.228608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.228882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.229642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:16.229949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.231659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.231926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.232580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.233742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.233742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.233938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:16.234183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.235803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.247210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 779/1566] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:16.254688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 788/1567] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:16.254896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.254987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.259461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec7b77-467f-44c4-a18f-9cfe79a46025 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.264704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.265066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636adcf-2512-4b8f-82df-aba092499fc1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.265373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.265686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.265963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.266698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.268301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.269939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.270609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.270888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.271157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.271956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.272970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:31:16.272970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:31:16.272970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:31:16.274372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.279126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 780/1568] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.280108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.292415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f0d67-2353-4741-bfd5-7cd813f0ad97 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.297902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.298555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.298839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.302163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 789/1569] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.302316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.302316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:16.302486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.302581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.302581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.309527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b568-1ad1-4928-8431-b57e6165ec9e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.313955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 781/1570] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:16.314753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.319199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.320051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.320336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.320695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.321571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:16.321884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.323742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.327965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ead6d-b032-4db1-bc88-9be7d286136d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.332462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.333113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.333430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.333704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.334392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.336298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.341715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 790/1571] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:16.341715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.353975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7024a9e-f9b7-4ebd-bd07-8e9e0fa7a325 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.358059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 782/1572] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 283 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.359139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.361742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.362499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.362785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.363064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.363795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.365791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.370431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937f4a-6d71-47be-81f3-5f8e93a590a6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.375476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.376356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.376615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.377003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.377710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.379646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.388364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 791/1573] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 296 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.389294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.399373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111392-7cc1-41bc-ab9b-e5c6a93e3f83 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.404622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.405630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.405949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.406232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.406922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.408846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.412627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 783/1574] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.412627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.440137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 792/1575] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.440137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.476751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051af1-afb0-474a-ab27-08f7dfba2cd5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.480943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:16.501055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1b0c1-acc3-4208-a6a0-06e1cd56e46d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.505818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.506509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.506790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.507066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.507794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.509615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.520152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 793/1576] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.521220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.528438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:16.532702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd38c1-95fb-47ef-afac-8709a12a651e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.537362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.538208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.538350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.538561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.538833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] PATH_INFO: `/v3/projects/959c5c84bd724d2688f435b15b41f1c8/users/14da69798e344e70ba64bf8187523e45/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.538941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.539112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.539410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.540109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.542369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.553951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=959c5c84bd724d2688f435b15b41f1c8, user_id=14da69798e344e70ba64bf8187523e45, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.555164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:31:16.555164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:31:16.555164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:31:16.557122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.573156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 794/1577] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.573156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.573500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 784/1578] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:16 2022] PUT /identity/v3/projects/959c5c84bd724d2688f435b15b41f1c8/users/14da69798e344e70ba64bf8187523e45/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:16.573500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.635266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562606a-6d1e-4030-b7ce-2e25dd7e9d80 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.635490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620a765-ea66-4697-8163-83b99b1bff76 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.640007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.640183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.640699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.640823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.640977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.641075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.641248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.641359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.641991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.642106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.643588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.643976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.653330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 785/1579] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.654235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.663984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c88da4-9abf-4f8c-94e3-b1802a58da62 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.668795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.669456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.669773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.670044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.674013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 795/1580] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.674168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.674280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.674383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.685512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 786/1581] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.685512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.693598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a4953-5132-41e0-a60f-10bf1842e1e8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.737886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22e98-e976-46fc-ae88-35f54832cbdb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.742564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.743230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.743511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.743786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.744710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.746432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.751057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.751833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.752119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.752394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.753068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.755062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.769931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 796/1582] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.770742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.780296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c141a2-2a4f-4369-b338-7a18ec126219 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.780999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 787/1583] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.784749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.784872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f1017-3b08-4752-9449-5b7a5df75424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.785427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1017-3b08-4752-9449-5b7a5df75424 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.785722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1017-3b08-4752-9449-5b7a5df75424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.785999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1017-3b08-4752-9449-5b7a5df75424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.786755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f1017-3b08-4752-9449-5b7a5df75424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.788359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f1017-3b08-4752-9449-5b7a5df75424 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.791224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:16.795813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3add55-8cd2-4387-bca0-5e68f4d1732d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.799060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 797/1584] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.802650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.802985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.803271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.803546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.803639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.803917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.804197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.804886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.806858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.821770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:16.823018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:16.828447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 788/1585] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.829372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.834278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 798/1586] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:16.835021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.839129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e06d7-fab4-4946-8e18-fb4cbfcb2bb0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.844411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.845080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.845216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39ae23-29d9-4d33-823f-c4067d5e35a2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.845363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.845697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.846759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.849337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.850398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.851066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.851347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.851603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.852293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.854253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.867911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 789/1587] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.867911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.877557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2c63c-c42e-4c57-8757-fe68b5dbc197 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.882180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.882839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.883135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.883390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.884116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.884835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 799/1588] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:16 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:16.885772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.886223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.896037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22f062-24f1-472f-b2ee-98bd731219d9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.902046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 790/1589] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.902046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.904629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.905706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.906075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.906795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.907087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.908698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.910028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660beada-3665-406c-81a0-0a016b8625e2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.914562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.915212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.915495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.915769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.916807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.918406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.922300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 800/1590] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.922300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.930020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c282df-a838-4a05-a727-450ce7dcbd5c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.934737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 791/1591] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.934737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.934950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.935516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.935826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.936131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] PATH_INFO: `/v3/projects/7269a91653d744c7a56d2188f1d10ab5/users/946fa7f26f644b58bce754c95e4298ec/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.943032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518c0-bd49-47e7-9101-84e653f4a4f2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:16.945215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:16.948525 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:16.949184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:16.949456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:16.949743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:16.950521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.952263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.953835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7269a91653d744c7a56d2188f1d10ab5, user_id=946fa7f26f644b58bce754c95e4298ec, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:16.956563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:16.966144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 792/1592] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:16.966144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:16.971206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 801/1593] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:16 2022] PUT /identity/v3/projects/7269a91653d744c7a56d2188f1d10ab5/users/946fa7f26f644b58bce754c95e4298ec/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:16.972050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.026851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f719160-b6e3-4d41-b931-321b98bddd5b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.034211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047219a-9311-4e46-9ba8-5dde6b3552b7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.038883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.039768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.040045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.040318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] PATH_INFO: `/v3/projects/b7b0a1a9f70a4a448c77cc45b8edf3e5/users/f8f99b7314be4820a7dcf0cce9a9050c/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.055265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b0a1a9f70a4a448c77cc45b8edf3e5, user_id=f8f99b7314be4820a7dcf0cce9a9050c, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.057981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.071702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 802/1594] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:16 2022] PUT /identity/v3/projects/b7b0a1a9f70a4a448c77cc45b8edf3e5/users/f8f99b7314be4820a7dcf0cce9a9050c/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:17.072610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.088738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.089395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.089708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.089982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.099509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.100935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.110543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 793/1595] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:17.111389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.121337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24983190-3a56-4e6b-8ab0-4a1dc6f38918 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.143747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3b1d9-baa9-4a38-8ec1-b6ff20fe5c91 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.148706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.149404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.149695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.149972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.150691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.152328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.168951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 803/1596] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:17.168951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.178322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a75ab-7627-4d42-aa3b-40803cebd4df admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.183752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.184415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.184697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.184973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.185820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.187669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.194171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13b64c-abd8-4be1-ab34-9b3daa777cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.195289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13b64c-abd8-4be1-ab34-9b3daa777cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.195642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13b64c-abd8-4be1-ab34-9b3daa777cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.196026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13b64c-abd8-4be1-ab34-9b3daa777cbd admin admin] PATH_INFO: `/v3/projects/63972a518bb048d19dd070eb9ac85445/users/2d3b91bfd0304cc793e3ad52c9b9e865/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.200720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 804/1597] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:17.200895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.202235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:17.212960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270805d2-95ec-4322-98b0-99387fca00d3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.218991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.219290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13b64c-abd8-4be1-ab34-9b3daa777cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=63972a518bb048d19dd070eb9ac85445, user_id=2d3b91bfd0304cc793e3ad52c9b9e865, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.219678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.220054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.220451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.221274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.222070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13b64c-abd8-4be1-ab34-9b3daa777cbd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.223070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.235893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 805/1598] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:17.236983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.238356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 794/1599] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:17 2022] PUT /identity/v3/projects/63972a518bb048d19dd070eb9ac85445/users/2d3b91bfd0304cc793e3ad52c9b9e865/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:17.240134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.240425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.240707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.249813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102706ab-fa2c-41d5-a733-0e397a6bef73 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.285613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] MFA Rules not processed for user `946fa7f26f644b58bce754c95e4298ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:17.287135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:17.297267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:17.326219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c0c5-b72e-48e0-87c9-b33575e13811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.327091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c0c5-b72e-48e0-87c9-b33575e13811 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.327994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c0c5-b72e-48e0-87c9-b33575e13811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.328302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c0c5-b72e-48e0-87c9-b33575e13811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.329053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c0c5-b72e-48e0-87c9-b33575e13811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.331227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c0c5-b72e-48e0-87c9-b33575e13811 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.347298 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 806/1600] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:17.348789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.349085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.349554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.376851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 795/1601] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:17.381988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.383913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] MFA Rules not processed for user `f8f99b7314be4820a7dcf0cce9a9050c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:17.385427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:17.398027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4a581-f179-45bb-b105-a5ce1b74f135 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.405621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.406756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.407065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.407397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.408148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.409938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.423168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 796/1602] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:17.424022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.434553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27d6d3-7cb1-4da9-8d8e-6c0749f38080 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.439437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab69ef6-d48d-46c0-873c-ef8263a11a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.440338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69ef6-d48d-46c0-873c-ef8263a11a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.440618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69ef6-d48d-46c0-873c-ef8263a11a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.440890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab69ef6-d48d-46c0-873c-ef8263a11a41 admin admin] PATH_INFO: `/v3/projects/f6ab199409614959939e8c00baa40d08/users/49468ccc681c4d7293a2c18d6535a1ad/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.456095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab69ef6-d48d-46c0-873c-ef8263a11a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ab199409614959939e8c00baa40d08, user_id=49468ccc681c4d7293a2c18d6535a1ad, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.458837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab69ef6-d48d-46c0-873c-ef8263a11a41 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.475338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 807/1603] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:17.476563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.485471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 797/1604] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:17 2022] PUT /identity/v3/projects/f6ab199409614959939e8c00baa40d08/users/49468ccc681c4d7293a2c18d6535a1ad/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:17.485471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.485471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.485471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.498528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:17.499964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:17.515103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa4ff8-f19e-4b5d-a0d8-d84e9415e76a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.570097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 798/1605] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:17.571729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.591703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bec7af-5b37-402f-ace6-f85092fd249d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.597520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d17ffc-f4d4-40e0-85a3-8cfb462cf454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.598591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17ffc-f4d4-40e0-85a3-8cfb462cf454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.598961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17ffc-f4d4-40e0-85a3-8cfb462cf454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.599242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17ffc-f4d4-40e0-85a3-8cfb462cf454 admin admin] PATH_INFO: `/v3/projects/40b09e85301c4e349b6714b13ae2e955/users/52ceb5b9477e431ba4a01adf2f3f485a/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.603322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.604344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.604727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.605121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] PATH_INFO: `/v3/projects/ff58bd7d53e149b19884ab8581d5bbfb/users/afce0f3286a2403cb3d216b5b8782c7b/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.616638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17ffc-f4d4-40e0-85a3-8cfb462cf454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b09e85301c4e349b6714b13ae2e955, user_id=52ceb5b9477e431ba4a01adf2f3f485a, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.619711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17ffc-f4d4-40e0-85a3-8cfb462cf454 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.625993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff58bd7d53e149b19884ab8581d5bbfb, user_id=afce0f3286a2403cb3d216b5b8782c7b, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.629197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.635974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 799/1606] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:17 2022] PUT /identity/v3/projects/40b09e85301c4e349b6714b13ae2e955/users/52ceb5b9477e431ba4a01adf2f3f485a/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:17.638076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5b85c-3b74-4f06-b3cb-1884190649a0 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.638641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5b85c-3b74-4f06-b3cb-1884190649a0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.639759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5b85c-3b74-4f06-b3cb-1884190649a0 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.646649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 800/1607] 10.222.0.24 () {54 vars in 961 bytes} [Thu May 19 13:31:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:17.646649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5b85c-3b74-4f06-b3cb-1884190649a0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.656999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 808/1608] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:17 2022] PUT /identity/v3/projects/ff58bd7d53e149b19884ab8581d5bbfb/users/afce0f3286a2403cb3d216b5b8782c7b/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:17.658182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.676415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:17.726920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:17.730954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5b85c-3b74-4f06-b3cb-1884190649a0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.736151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60453b7-16cd-4f04-9cbb-da0e3482d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.737131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60453b7-16cd-4f04-9cbb-da0e3482d612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.737732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60453b7-16cd-4f04-9cbb-da0e3482d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.738015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60453b7-16cd-4f04-9cbb-da0e3482d612 admin admin] PATH_INFO: `/v3/projects/6c7967c1e3ad47cc9e2fbd3de2c673b5/users/2b7e594fef204cf48bde8b68b7c6f2f8/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.739332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ac6c-d956-4f82-9bf3-3bdec77e0df7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.745167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.746109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.746250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.746512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.747314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.749516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.758898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60453b7-16cd-4f04-9cbb-da0e3482d612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7967c1e3ad47cc9e2fbd3de2c673b5, user_id=2b7e594fef204cf48bde8b68b7c6f2f8, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.763420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60453b7-16cd-4f04-9cbb-da0e3482d612 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.781588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 801/1609] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:17 2022] PUT /identity/v3/projects/6c7967c1e3ad47cc9e2fbd3de2c673b5/users/2b7e594fef204cf48bde8b68b7c6f2f8/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:17.783208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e1e0-8778-420d-94fc-86558d97473b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.783561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e1e0-8778-420d-94fc-86558d97473b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.783897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e1e0-8778-420d-94fc-86558d97473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.787675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 809/1610] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:17.789220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:17.835812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9d43e1e0-8778-420d-94fc-86558d97473b None None] MFA Rules not processed for user `49468ccc681c4d7293a2c18d6535a1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:17.837304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9d43e1e0-8778-420d-94fc-86558d97473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:17.871464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c169d6-7b15-48c1-bcd8-8e2b546712a7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:17.877083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded4bb0-770c-4169-9cfa-73b646a738bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:17.877912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4bb0-770c-4169-9cfa-73b646a738bb service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.878296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4bb0-770c-4169-9cfa-73b646a738bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.878671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded4bb0-770c-4169-9cfa-73b646a738bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.931126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 802/1611] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:17.948119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75850f-927f-426c-b765-494baff39543 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.948978 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75850f-927f-426c-b765-494baff39543 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.950006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75850f-927f-426c-b765-494baff39543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.972794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ca75850f-927f-426c-b765-494baff39543 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:17.974137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ca75850f-927f-426c-b765-494baff39543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:17.974489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4bb0-770c-4169-9cfa-73b646a738bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:17.976975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded4bb0-770c-4169-9cfa-73b646a738bb service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:17.990679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 810/1612] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:17.992361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:17.992512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:17.992864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:17.997738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 803/1613] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:18.001480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1e7b8-32d5-46b5-be92-ff55f3b3884a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.001798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1e7b8-32d5-46b5-be92-ff55f3b3884a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.002087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1e7b8-32d5-46b5-be92-ff55f3b3884a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.017696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:18.019664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:18.037641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dfa1e7b8-32d5-46b5-be92-ff55f3b3884a None None] MFA Rules not processed for user `afce0f3286a2403cb3d216b5b8782c7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:18.039053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dfa1e7b8-32d5-46b5-be92-ff55f3b3884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:18.125774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 811/1614] 10.222.0.24 () {58 vars in 1058 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:18.126911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.135367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 804/1615] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:18.137313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.137625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.137908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.144239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46f14a-1d04-4a5b-b5e4-ae4306961fc8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.154860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.155549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.155879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.156157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.157362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.159436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:18.159964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.171812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] MFA Rules not processed for user `2b7e594fef204cf48bde8b68b7c6f2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:18.173916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:18.192398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 812/1616] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:18 2022] POST /identity/v3/users => generated 288 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:18.193241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.206511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:18.304054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 805/1617] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:18.305455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.307383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:18.323857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e6fb-e166-4d2f-a546-59868a07a1f5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.328874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a7f31-b77e-48ec-b6c6-91949602c53d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.334611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.334935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.335916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.336229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.336555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.339203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.339745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.341335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.353332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.355082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.368985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 806/1618] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:18.369901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.383403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.388334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51c9e9-8d12-4282-a34c-32e23d62bc62 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.391255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.396678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.397358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.397668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.397946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.453177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 813/1619] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:18.454197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.507151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1e3ee-7511-464a-afd0-ee904bff9d89 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.523769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.525655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.526151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6b8cb-a156-4461-adc1-0122430c19f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.526835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6b8cb-a156-4461-adc1-0122430c19f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.527147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6b8cb-a156-4461-adc1-0122430c19f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.527642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6b8cb-a156-4461-adc1-0122430c19f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.528454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6b8cb-a156-4461-adc1-0122430c19f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.530601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6b8cb-a156-4461-adc1-0122430c19f9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.551909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 807/1620] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:18.558335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.594981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98d9b6-2bc6-4915-973f-cf9147d29d79 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.611883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.614035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.614607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.615092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.616100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.620444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.645540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 814/1621] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:18 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:18.646917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a03edf-94e0-48ec-b6eb-a9ac8b55e8de None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.647313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a03edf-94e0-48ec-b6eb-a9ac8b55e8de None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.648976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a03edf-94e0-48ec-b6eb-a9ac8b55e8de None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.653510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 815/1622] 10.222.0.24 () {54 vars in 961 bytes} [Thu May 19 13:31:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:18.654795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a03edf-94e0-48ec-b6eb-a9ac8b55e8de None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.658259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 808/1623] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:18.662527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.692893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061b1f-099f-4b0a-9062-12b47226eec4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.837283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a03edf-94e0-48ec-b6eb-a9ac8b55e8de None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:18.840270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.841366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.841775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.842466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] PATH_INFO: `/v3/projects/e3e1e2f24ef04a03a8d7c0c556f4656a/users/883782d21ecd462ea2a0170ddb5c510a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.856330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb85ad4-8cdd-445c-9518-b6730ac83ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:18.857033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb85ad4-8cdd-445c-9518-b6730ac83ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.857320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb85ad4-8cdd-445c-9518-b6730ac83ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.857806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb85ad4-8cdd-445c-9518-b6730ac83ee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.858735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb85ad4-8cdd-445c-9518-b6730ac83ee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.860661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb85ad4-8cdd-445c-9518-b6730ac83ee0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.891255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e1e2f24ef04a03a8d7c0c556f4656a, user_id=883782d21ecd462ea2a0170ddb5c510a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:18.898975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:18.921080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:18.924908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 816/1624] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:18 2022] POST /identity/v3/users => generated 308 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:18.926793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:18.927094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:18.927543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:18.930597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 809/1625] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:18 2022] PUT /identity/v3/projects/e3e1e2f24ef04a03a8d7c0c556f4656a/users/883782d21ecd462ea2a0170ddb5c510a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:18.931515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:18.950314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:18.952370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:19.107666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b7cda-cc23-4acc-b240-0a25314d4195 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.120848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.121687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.122090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.122519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.123981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:19.131265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:19.152158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 810/1626] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:19.153783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:19.181582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 817/1627] 10.222.0.24 () {58 vars in 1058 bytes} [Thu May 19 13:31:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:19.182562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:19.202696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b87b7-6a94-4ef9-a568-954de8db5f07 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.205962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:19.222166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.222476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.222951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.223306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.224290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:19.228768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:19.264018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ebdaa-3fcf-43a6-9b36-4eb9c60daf8a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.265496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 811/1628] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:19.266586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:19.281765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b7343-ddbf-41b5-9dc3-6c3db6e5e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.282695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b7343-ddbf-41b5-9dc3-6c3db6e5e011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.282983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b7343-ddbf-41b5-9dc3-6c3db6e5e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.288803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b7343-ddbf-41b5-9dc3-6c3db6e5e011 admin admin] PATH_INFO: `/v3/projects/82adef0771bc4f239b3a927180d065e7/users/fa7b03ad03da468a979e0f20c5c8d63e/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.298582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fc436-bf74-4db2-a0b9-41b09e6eaf9d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.312011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.312823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.313193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.313622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.401446 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b7343-ddbf-41b5-9dc3-6c3db6e5e011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82adef0771bc4f239b3a927180d065e7, user_id=fa7b03ad03da468a979e0f20c5c8d63e, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:19.403325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:19.405513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:19.413170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b7343-ddbf-41b5-9dc3-6c3db6e5e011 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:19.419867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 812/1629] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:19.421675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:19.446972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d64de-95ba-4dcb-b9c9-a2a87f7eaa24 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.452832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 818/1630] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:19 2022] PUT /identity/v3/projects/82adef0771bc4f239b3a927180d065e7/users/fa7b03ad03da468a979e0f20c5c8d63e/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:19.461356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe4b6-75fd-4a5d-8628-33f6a828a267 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.462287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe4b6-75fd-4a5d-8628-33f6a828a267 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.465665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfe4b6-75fd-4a5d-8628-33f6a828a267 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.468124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 819/1631] 10.222.0.24 () {54 vars in 912 bytes} [Thu May 19 13:31:19 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:19.470204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfe4b6-75fd-4a5d-8628-33f6a828a267 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:19.567055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffe751-6399-432e-ba1d-a9871035b1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.568122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffe751-6399-432e-ba1d-a9871035b1a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.568494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffe751-6399-432e-ba1d-a9871035b1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.568862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffe751-6399-432e-ba1d-a9871035b1a8 admin admin] PATH_INFO: `/v3/projects/e3e1e2f24ef04a03a8d7c0c556f4656a/users/883782d21ecd462ea2a0170ddb5c510a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.574960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffe751-6399-432e-ba1d-a9871035b1a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e1e2f24ef04a03a8d7c0c556f4656a, user_id=883782d21ecd462ea2a0170ddb5c510a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:19.578272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffe751-6399-432e-ba1d-a9871035b1a8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:19.602450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 813/1632] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:19 2022] PUT /identity/v3/projects/e3e1e2f24ef04a03a8d7c0c556f4656a/users/883782d21ecd462ea2a0170ddb5c510a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:19.606307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca7b8b-6729-4f4d-aa45-c67ddc9c8537 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.606813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca7b8b-6729-4f4d-aa45-c67ddc9c8537 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.608012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca7b8b-6729-4f4d-aa45-c67ddc9c8537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.645278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f9ca7b8b-6729-4f4d-aa45-c67ddc9c8537 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:19.648024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfe4b6-75fd-4a5d-8628-33f6a828a267 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.648518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f9ca7b8b-6729-4f4d-aa45-c67ddc9c8537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:19.670022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:19.804749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 814/1633] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:31:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:19.807660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.808070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.808821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.815470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.816375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.816889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.817266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:19.818333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:19.822537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:19.873107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 820/1634] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:19.875354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] MFA Rules not processed for user `883782d21ecd462ea2a0170ddb5c510a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:19.877058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:19.878905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:19.985224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b55672-e80e-4b0f-904a-6e69f498a91d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:19.994227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679d814-2841-402d-89fa-a83359f6196b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:19.995418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d814-2841-402d-89fa-a83359f6196b service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:19.995710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d814-2841-402d-89fa-a83359f6196b service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:19.996035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679d814-2841-402d-89fa-a83359f6196b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.018713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679d814-2841-402d-89fa-a83359f6196b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:20.020200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679d814-2841-402d-89fa-a83359f6196b service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:20.030026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 815/1635] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:20.031196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:20.043928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf1e20-e7f5-4178-b21e-8a25e6fb875d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:20.056528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce0dec-72ba-4143-a1c5-5bc0bcbb476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:20.058032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce0dec-72ba-4143-a1c5-5bc0bcbb476b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.058311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce0dec-72ba-4143-a1c5-5bc0bcbb476b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.058424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce0dec-72ba-4143-a1c5-5bc0bcbb476b admin admin] PATH_INFO: `/v3/projects/82adef0771bc4f239b3a927180d065e7/users/fa7b03ad03da468a979e0f20c5c8d63e/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.062822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 821/1636] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:20.064425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da923fc-3001-4996-a016-c17f2a819416 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.064722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da923fc-3001-4996-a016-c17f2a819416 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.065003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da923fc-3001-4996-a016-c17f2a819416 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.067080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce0dec-72ba-4143-a1c5-5bc0bcbb476b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82adef0771bc4f239b3a927180d065e7, user_id=fa7b03ad03da468a979e0f20c5c8d63e, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:20.070509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce0dec-72ba-4143-a1c5-5bc0bcbb476b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:20.078113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 822/1637] 10.222.0.24 () {54 vars in 912 bytes} [Thu May 19 13:31:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:20.079117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da923fc-3001-4996-a016-c17f2a819416 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:20.096643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 816/1638] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:20 2022] PUT /identity/v3/projects/82adef0771bc4f239b3a927180d065e7/users/fa7b03ad03da468a979e0f20c5c8d63e/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:20.103665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da923fc-3001-4996-a016-c17f2a819416 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:20.108432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e607dc3-df74-436b-add9-a9e45ac87890 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.108999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e607dc3-df74-436b-add9-a9e45ac87890 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.109293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e607dc3-df74-436b-add9-a9e45ac87890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.153062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6e607dc3-df74-436b-add9-a9e45ac87890 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:20.154340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6e607dc3-df74-436b-add9-a9e45ac87890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:20.170688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:20.211130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:20.211933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.212268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.212552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.353026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 817/1639] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:31:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:20.356307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.357534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.358381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.381440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:20.384293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:20.413856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 823/1640] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:20 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:20.414319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:20.415232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:20.432358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] MFA Rules not processed for user `fa7b03ad03da468a979e0f20c5c8d63e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:20.436548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:20.560486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 818/1641] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:20.560486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:20.574103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0750e1-56d6-4e84-a282-708b584614f2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:20.578607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a79c9-7f01-4884-8a6b-bc7cc40c6626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:20.579461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a79c9-7f01-4884-8a6b-bc7cc40c6626 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.579614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a79c9-7f01-4884-8a6b-bc7cc40c6626 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.579875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a79c9-7f01-4884-8a6b-bc7cc40c6626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.584926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4b6c-8ad2-4493-ab24-0b7cc3e2318e service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:20.596742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a79c9-7f01-4884-8a6b-bc7cc40c6626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:20.598311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a79c9-7f01-4884-8a6b-bc7cc40c6626 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:20.602023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:20.603776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.604366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.604759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] PATH_INFO: `/v3/users/946fa7f26f644b58bce754c95e4298ec` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.606995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=946fa7f26f644b58bce754c95e4298ec)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:20.611734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 819/1642] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:20.612227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:20.613152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.613476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.614002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.677917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] MFA Rules not processed for user `52ceb5b9477e431ba4a01adf2f3f485a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:20.679132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:20.731900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946fa7f26f644b58bce754c95e4298ec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:20.762276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946fa7f26f644b58bce754c95e4298ec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:20.767264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946fa7f26f644b58bce754c95e4298ec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:20.769535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946fa7f26f644b58bce754c95e4298ec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:20.789337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a85fc071-0694-47a8-ab9d-a1ce39ee21b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946fa7f26f644b58bce754c95e4298ec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:20.797096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:20.797868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 820/1643] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:20.800400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:20.806581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:20.851660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 824/1644] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:20 2022] DELETE /identity/v3/users/946fa7f26f644b58bce754c95e4298ec => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:20.853110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf8a7d-0716-40a7-a025-6b4e6b4b9813 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.853427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf8a7d-0716-40a7-a025-6b4e6b4b9813 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.853711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf8a7d-0716-40a7-a025-6b4e6b4b9813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:20.862676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f87c7c-feb5-4f45-b65a-8feed2e5dee2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:20.897596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cbdf8a7d-0716-40a7-a025-6b4e6b4b9813 None None] MFA Rules not processed for user `52ceb5b9477e431ba4a01adf2f3f485a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:20.897853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cbdf8a7d-0716-40a7-a025-6b4e6b4b9813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:20.966038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:20.986584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:20.986874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:20.987189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:21.015150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:21.036857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 825/1645] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:21.038307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:21.038602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:21.038892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:21.065444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:21.085319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:21.089368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:21.153939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:21.157474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:21.180119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 821/1646] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:21.180990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:21.184389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:21.210160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 826/1647] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:21.245194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:21.246925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:21.338555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8d9f2-27c4-4e07-935f-6dc04c4d77e6 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:21.347866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:21.348849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:21.349275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:21.349632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:21.360021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:21.361926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:21.382310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 822/1648] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:21.390340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81691cd-f513-41f3-81e2-aa06750721a9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:21.395992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aad12f-2a8f-41ad-9328-07a912c0f367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:21.396950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aad12f-2a8f-41ad-9328-07a912c0f367 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:21.397282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aad12f-2a8f-41ad-9328-07a912c0f367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:21.397592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aad12f-2a8f-41ad-9328-07a912c0f367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:21.423407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aad12f-2a8f-41ad-9328-07a912c0f367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:21.426489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aad12f-2a8f-41ad-9328-07a912c0f367 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:21.451459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 827/1649] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:21.708236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:21.709715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:21.734361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869bc4-c139-4e2a-99a8-1109a0f106ec service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:21.753801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:21.754666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:21.754955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:21.755280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] PATH_INFO: `/v3/users/afce0f3286a2403cb3d216b5b8782c7b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:21.761535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afce0f3286a2403cb3d216b5b8782c7b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:21.761535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: WARNING py.warnings [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:31:21.761535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: warnings.warn(msg) May 19 13:31:21.761535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: May 19 13:31:21.761535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:21.948097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afce0f3286a2403cb3d216b5b8782c7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:21.970698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afce0f3286a2403cb3d216b5b8782c7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:21.971382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afce0f3286a2403cb3d216b5b8782c7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:21.973008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afce0f3286a2403cb3d216b5b8782c7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:22.012690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afce0f3286a2403cb3d216b5b8782c7b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:22.054272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 823/1650] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:21 2022] DELETE /identity/v3/users/afce0f3286a2403cb3d216b5b8782c7b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:22.059162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:22.059670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:22.059774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:22.121665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:22.121665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:22.253283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:22.292913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 828/1651] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:22.332858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:22.334357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:22.409724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:22.521770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d11905-8280-4739-b036-2155822db3d7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:22.528085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9772f43-4c50-4ec7-ae6e-bedc5a406603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:22.533123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772f43-4c50-4ec7-ae6e-bedc5a406603 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:22.534198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772f43-4c50-4ec7-ae6e-bedc5a406603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:22.534607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9772f43-4c50-4ec7-ae6e-bedc5a406603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:22.542817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:22.545237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:22.549941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9772f43-4c50-4ec7-ae6e-bedc5a406603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:22.553184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9772f43-4c50-4ec7-ae6e-bedc5a406603 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:22.564709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c10ce0-0fa6-4926-bbf3-226437caaf14 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:22.570822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:22.572291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:22.572944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:22.573223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] PATH_INFO: `/v3/users/f8f99b7314be4820a7dcf0cce9a9050c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:22.574497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f99b7314be4820a7dcf0cce9a9050c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:22.577146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:22.581596 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 824/1652] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:22.867009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92307488-dec3-4e33-9dca-0d98c28ae332 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:22.867579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92307488-dec3-4e33-9dca-0d98c28ae332 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:22.867748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92307488-dec3-4e33-9dca-0d98c28ae332 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:22.869243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 825/1653] 10.222.0.24 () {54 vars in 912 bytes} [Thu May 19 13:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 13:31:22.876619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870314f0-4d64-474c-b8d7-acf904e65aa9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:22.877394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870314f0-4d64-474c-b8d7-acf904e65aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:22.877394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870314f0-4d64-474c-b8d7-acf904e65aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:22.944915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f99b7314be4820a7dcf0cce9a9050c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:22.992089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f99b7314be4820a7dcf0cce9a9050c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:22.996600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f99b7314be4820a7dcf0cce9a9050c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:22.999119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f99b7314be4820a7dcf0cce9a9050c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:23.015858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-870314f0-4d64-474c-b8d7-acf904e65aa9 None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:23.022737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-870314f0-4d64-474c-b8d7-acf904e65aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:23.024568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f99b7314be4820a7dcf0cce9a9050c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:23.045300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 826/1654] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:23.060260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 829/1655] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:22 2022] DELETE /identity/v3/users/f8f99b7314be4820a7dcf0cce9a9050c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:23.064919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:23.065212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:23.065514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:23.108319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:23.111439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:23.178140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:23.179853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:23.317944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 827/1656] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:23.330820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956924-84fc-45e6-990a-2a387688ce6f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:23.335710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:23.336732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:23.337186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:23.338466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:23.348554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:23.350458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:23.361035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 830/1657] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:23.372085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:23.374583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:23.403312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006721f-6c7f-4d6b-a0a6-7024e867ef54 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:23.409888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:23.410931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:23.411431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:23.411940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:23.425179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:23.427491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:23.440604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 828/1658] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:23.948755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:23.950692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:23.989901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d486e-911f-4e98-960e-c5da0b837b0e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:23.998947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:24.000133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:24.000428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:24.000703 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] PATH_INFO: `/v3/projects/7269a91653d744c7a56d2188f1d10ab5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:24.002644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7269a91653d744c7a56d2188f1d10ab5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:24.003743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 13:31:24.003743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: warnings.warn(msg) May 19 13:31:24.003743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: May 19 13:31:24.004958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:24.145626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:24.147792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:24.166595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7269a91653d744c7a56d2188f1d10ab5', 'request_id': 'req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.175707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:24.179912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc0aa0-df94-4d6f-91fd-dea75beb8637 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:24.189149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7269a91653d744c7a56d2188f1d10ab5', 'request_id': 'req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.213305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7269a91653d744c7a56d2188f1d10ab5', 'request_id': 'req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.254457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] The token cache is being invalidate because project 7269a91653d744c7a56d2188f1d10ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:24.254780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.264687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a12bc4a1-678f-4a02-8a6f-a763ab10cac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.270440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 831/1659] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:23 2022] DELETE /identity/v3/projects/7269a91653d744c7a56d2188f1d10ab5 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:24.274509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d497a31-d8fa-4e81-98df-383bbd30f073 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:24.274828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d497a31-d8fa-4e81-98df-383bbd30f073 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:24.275112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d497a31-d8fa-4e81-98df-383bbd30f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:24.343896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:24.345074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0d497a31-d8fa-4e81-98df-383bbd30f073 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:24.345297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:24.345946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:24.346383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] PATH_INFO: `/v3/projects/ff58bd7d53e149b19884ab8581d5bbfb` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:24.347873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff58bd7d53e149b19884ab8581d5bbfb)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:24.348030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d497a31-d8fa-4e81-98df-383bbd30f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:24.350332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:24.381520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 832/1660] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:24.384418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:24.384803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:24.385431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:24.421646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:24.425015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:24.427257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:24.458449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff58bd7d53e149b19884ab8581d5bbfb', 'request_id': 'req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.486183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff58bd7d53e149b19884ab8581d5bbfb', 'request_id': 'req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.497845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff58bd7d53e149b19884ab8581d5bbfb', 'request_id': 'req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.533286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] The token cache is being invalidate because project ff58bd7d53e149b19884ab8581d5bbfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:24.535336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.536568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.543378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 829/1661] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:24 2022] DELETE /identity/v3/projects/ff58bd7d53e149b19884ab8581d5bbfb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:24.549424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:24.551851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:24.616466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 833/1662] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:24.617787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:24.620993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:24.650212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bcbce-d35b-460d-a036-a6b16576fff9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:24.664665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:24.665866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:24.666634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:24.667110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] PATH_INFO: `/v3/projects/b7b0a1a9f70a4a448c77cc45b8edf3e5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:24.673857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b0a1a9f70a4a448c77cc45b8edf3e5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:24.676299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:24.679531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aab5c-e2ba-4cbd-b24b-a1b62ce47868 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:24.693575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:24.694235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:24.694652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:24.694992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] PATH_INFO: `/v3/users/fa7b03ad03da468a979e0f20c5c8d63e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:24.696997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7b03ad03da468a979e0f20c5c8d63e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:24.700147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:24.798750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b0a1a9f70a4a448c77cc45b8edf3e5', 'request_id': 'req-7c9ad426-4add-4ccd-9498-25ef1637a85a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.824682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b0a1a9f70a4a448c77cc45b8edf3e5', 'request_id': 'req-7c9ad426-4add-4ccd-9498-25ef1637a85a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.836055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b0a1a9f70a4a448c77cc45b8edf3e5', 'request_id': 'req-7c9ad426-4add-4ccd-9498-25ef1637a85a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.854405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7b03ad03da468a979e0f20c5c8d63e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.876818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] The token cache is being invalidate because project b7b0a1a9f70a4a448c77cc45b8edf3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:24.880968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.882561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7b03ad03da468a979e0f20c5c8d63e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.883319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.885504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7b03ad03da468a979e0f20c5c8d63e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.888452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7b03ad03da468a979e0f20c5c8d63e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:24.893074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 834/1663] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:24 2022] DELETE /identity/v3/projects/b7b0a1a9f70a4a448c77cc45b8edf3e5 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:24.894079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:24.903446 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:24.970271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7b03ad03da468a979e0f20c5c8d63e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:25.022627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 830/1664] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:24 2022] DELETE /identity/v3/users/fa7b03ad03da468a979e0f20c5c8d63e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:25.023883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.025499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:25.105272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ad426-4add-4ccd-9498-25ef1637a85a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.116040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:25.180259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:25.268681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a8475-3158-42ac-98e5-7b4dd3164194 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.287307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.287999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.288292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.288600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.288938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.289392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:25.289763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.290451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.290858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.291711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4c262-1bd5-456e-b084-2662fdce0ea4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.291820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] PATH_INFO: `/v3/users/883782d21ecd462ea2a0170ddb5c510a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.293196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=883782d21ecd462ea2a0170ddb5c510a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.298147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.323112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 835/1665] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:25.324598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2fa95-2807-45ba-a4e4-6838b364a493 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.324888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2fa95-2807-45ba-a4e4-6838b364a493 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.325171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2fa95-2807-45ba-a4e4-6838b364a493 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.335906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 836/1666] 10.222.0.24 () {54 vars in 960 bytes} [Thu May 19 13:31:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:25.336825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2fa95-2807-45ba-a4e4-6838b364a493 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.339075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9df2fa95-2807-45ba-a4e4-6838b364a493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:25.372928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2fa95-2807-45ba-a4e4-6838b364a493 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.384702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ccab0-f1f6-42f6-a807-d4d8d46a35ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.385399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ccab0-f1f6-42f6-a807-d4d8d46a35ba admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.387392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ccab0-f1f6-42f6-a807-d4d8d46a35ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.387695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ccab0-f1f6-42f6-a807-d4d8d46a35ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.388639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ccab0-f1f6-42f6-a807-d4d8d46a35ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.394051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ccab0-f1f6-42f6-a807-d4d8d46a35ba admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.395738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883782d21ecd462ea2a0170ddb5c510a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:25.422441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883782d21ecd462ea2a0170ddb5c510a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:25.423617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883782d21ecd462ea2a0170ddb5c510a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:25.424540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883782d21ecd462ea2a0170ddb5c510a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:25.462176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883782d21ecd462ea2a0170ddb5c510a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:25.488380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 837/1667] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:25 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:25.495319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.495632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.495915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.516997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 831/1668] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:25 2022] DELETE /identity/v3/users/883782d21ecd462ea2a0170ddb5c510a => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:25.518052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.521248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:25.553768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] MFA Rules not processed for user `48ab1e052fff48bfbbc222528424b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:25.560392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:25.693628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad7f35-f691-4848-b0d3-873f422a3f9e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.704770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.705493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.705779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.706052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.706759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.710362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.730769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 838/1669] 10.222.0.24 () {58 vars in 1057 bytes} [Thu May 19 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:25.740918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.769178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29c38f-af45-441d-84ff-0fcd4a8770a4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.770063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 832/1670] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:25 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:25.771162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.783464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841d2d-7014-4552-9b0c-2fcd6b723b42 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.785545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.786447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.786741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.789327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.792195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.793507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.793796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.794116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.794271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:25.794812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.797193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.815908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 833/1671] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:25.816850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.822885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.824535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635370e3-a0cb-4bed-9e8a-b7d79b1dd902 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.828157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2c5a5-85fe-4780-8013-48faa97ba237 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.833861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.834553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.834927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.835256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.836061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.838108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.857015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 834/1672] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:25.859095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 839/1673] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:25.859433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:25.879422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a73103-8c72-4664-a30b-83e67e658939 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:25.899882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:25.901031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.901350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:25.901904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] PATH_INFO: `/v3/projects/6a7ac23cb05647a4b5cbc1524c355228/users/e2e539dc5ad6438dbe988e5ccd4c89e3/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:25.945402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7ac23cb05647a4b5cbc1524c355228, user_id=e2e539dc5ad6438dbe988e5ccd4c89e3, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:25.948452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:25.984588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 835/1674] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:25 2022] PUT /identity/v3/projects/6a7ac23cb05647a4b5cbc1524c355228/users/e2e539dc5ad6438dbe988e5ccd4c89e3/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:25.996355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:25.999920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.001023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.068925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] MFA Rules not processed for user `e2e539dc5ad6438dbe988e5ccd4c89e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:26.070385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:26.257588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:26.296371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 840/1675] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:26.300742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:26.309437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:26.354663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:26.356411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:26.428216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401eb0aa-c315-4d7b-aaa7-963dbb2161b2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:26.446016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52264403-b4b6-432c-8ec7-59402b927d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:26.449025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52264403-b4b6-432c-8ec7-59402b927d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:26.449405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52264403-b4b6-432c-8ec7-59402b927d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.452030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52264403-b4b6-432c-8ec7-59402b927d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.453064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52264403-b4b6-432c-8ec7-59402b927d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:26.457984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52264403-b4b6-432c-8ec7-59402b927d5a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:26.475828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e144d6-be22-4c96-babe-a157cad15cb3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:26.481361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:26.482390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:26.482790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.483179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] PATH_INFO: `/v3/projects/82adef0771bc4f239b3a927180d065e7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.484755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82adef0771bc4f239b3a927180d065e7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:26.487955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:26.501159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 836/1676] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:26 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:26.503439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:26.503732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.504010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.548374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] MFA Rules not processed for user `14da69798e344e70ba64bf8187523e45`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:26.549875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:26.587725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82adef0771bc4f239b3a927180d065e7', 'request_id': 'req-fc749031-4fdd-4987-843b-680020947745'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.599920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82adef0771bc4f239b3a927180d065e7', 'request_id': 'req-fc749031-4fdd-4987-843b-680020947745'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.612155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82adef0771bc4f239b3a927180d065e7', 'request_id': 'req-fc749031-4fdd-4987-843b-680020947745'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.627087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] The token cache is being invalidate because project 82adef0771bc4f239b3a927180d065e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:26.627712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.628565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.631560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 841/1677] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:26 2022] DELETE /identity/v3/projects/82adef0771bc4f239b3a927180d065e7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:26.633087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:26.636908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:26.675940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 837/1678] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:26.684176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:26.684176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:26.814827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc749031-4fdd-4987-843b-680020947745 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:26.822148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213f473-e817-404e-a63e-e9d281c7e7d3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:26.825230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:26.826655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:26.827073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.827487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] PATH_INFO: `/v3/projects/e3e1e2f24ef04a03a8d7c0c556f4656a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.828295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f25974-09b1-4a75-adf6-ed303535346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:26.829183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e1e2f24ef04a03a8d7c0c556f4656a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:26.829530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f25974-09b1-4a75-adf6-ed303535346c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:26.829970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f25974-09b1-4a75-adf6-ed303535346c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.830335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f25974-09b1-4a75-adf6-ed303535346c admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.831403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f25974-09b1-4a75-adf6-ed303535346c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:26.831973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:26.833940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f25974-09b1-4a75-adf6-ed303535346c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:26.869485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 838/1679] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:26 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:26.871410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:26.871800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:26.872173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:26.876330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:26.905222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:26.918219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:26.921612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:26.949350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e1e2f24ef04a03a8d7c0c556f4656a', 'request_id': 'req-b00526da-c2e8-4e97-a8c9-b3181e199dec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.965678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e1e2f24ef04a03a8d7c0c556f4656a', 'request_id': 'req-b00526da-c2e8-4e97-a8c9-b3181e199dec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:26.980113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e1e2f24ef04a03a8d7c0c556f4656a', 'request_id': 'req-b00526da-c2e8-4e97-a8c9-b3181e199dec'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:27.034764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] The token cache is being invalidate because project e3e1e2f24ef04a03a8d7c0c556f4656a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:27.035269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:27.037759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:27.037869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 839/1680] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:27.039421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.040821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:27.045479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 842/1681] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:26 2022] DELETE /identity/v3/projects/e3e1e2f24ef04a03a8d7c0c556f4656a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:27.046635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.056495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:27.106058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00526da-c2e8-4e97-a8c9-b3181e199dec admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.111525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11b9af-30a3-4b1c-b835-157c0754a834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.112213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11b9af-30a3-4b1c-b835-157c0754a834 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.113032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11b9af-30a3-4b1c-b835-157c0754a834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.113032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11b9af-30a3-4b1c-b835-157c0754a834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.113553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11b9af-30a3-4b1c-b835-157c0754a834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.124190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11b9af-30a3-4b1c-b835-157c0754a834 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.158691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 843/1682] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:27.158691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a3f95-bd92-40f5-8625-1c14201aa760 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.158691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a3f95-bd92-40f5-8625-1c14201aa760 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.162163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a3f95-bd92-40f5-8625-1c14201aa760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.216875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2b9a3f95-bd92-40f5-8625-1c14201aa760 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:27.218414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b9a3f95-bd92-40f5-8625-1c14201aa760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:27.236771 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 844/1683] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:27.238425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.239454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.240667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.289664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:27.291377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:27.320132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 845/1684] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:27.320132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.332983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0c222-9d83-44dc-8c6b-ee7fc8938d26 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.349949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.350931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.351281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.351626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] PATH_INFO: `/v3/projects/4f0120f6bbdc49a6b5989b906dc35af8/users/8033e746642d4ad7bc04ae83ee469b1d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.358452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc80b4-d275-44c9-82ee-7ec9023da4f7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.367858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.368568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.386922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.387302 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.393835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:27.401838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0120f6bbdc49a6b5989b906dc35af8, user_id=8033e746642d4ad7bc04ae83ee469b1d, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.407741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.441374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 846/1685] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:27 2022] PUT /identity/v3/projects/4f0120f6bbdc49a6b5989b906dc35af8/users/8033e746642d4ad7bc04ae83ee469b1d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:27.442655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.568093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.571203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744ca13-ce91-4518-8f84-97a67a9816c4 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.595671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9f98a-2c93-4d3d-8492-d17a525b8336 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.598571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 840/1686] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:27.601660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.603413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.603733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.604421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.605261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.605644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.606015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.606839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:27.607396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.609963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.631276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 847/1687] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:27.632445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.648561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da72333-95d8-4086-b60f-d84792e775fc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.658214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.658945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.660059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.660557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.661677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.663582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.682821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:27.688352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:27.706047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 848/1688] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:27.707160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.713079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 841/1689] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:27.716195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.739330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86454dc-b177-4889-9c8e-d38329f7aeb3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.740960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa1497-865a-4ae6-a4ae-e6ff952ce9ef admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:27.745722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.747083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.747459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.749801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] PATH_INFO: `/v3/projects/4f0120f6bbdc49a6b5989b906dc35af8/users/8033e746642d4ad7bc04ae83ee469b1d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.761309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0120f6bbdc49a6b5989b906dc35af8, user_id=8033e746642d4ad7bc04ae83ee469b1d, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.764770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.770798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07173f-7eb8-49d3-90ab-e61fc603cd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:27.771485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07173f-7eb8-49d3-90ab-e61fc603cd1e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.771759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07173f-7eb8-49d3-90ab-e61fc603cd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.772035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07173f-7eb8-49d3-90ab-e61fc603cd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.772738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07173f-7eb8-49d3-90ab-e61fc603cd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:27.774714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07173f-7eb8-49d3-90ab-e61fc603cd1e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:27.829649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 842/1690] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:27 2022] PUT /identity/v3/projects/4f0120f6bbdc49a6b5989b906dc35af8/users/8033e746642d4ad7bc04ae83ee469b1d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:27.833180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:27.887259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 849/1691] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:27 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:27.898590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:27.900001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:27.901833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:27.914244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:28.029496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965df5-902a-4686-9f47-7ad2ab0b3469 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.039906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] MFA Rules not processed for user `8033e746642d4ad7bc04ae83ee469b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:28.042421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:28.082232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:28.145351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.146094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.147344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.147344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.147851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:28.148207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.149989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.185269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 843/1692] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:28.185269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.209344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce743914-8792-4fc4-a5d6-46f411915c24 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.227859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 850/1693] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:28.228705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.235731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.236704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.237121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.238064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.238916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.247026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.257907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d795d4-0045-4d52-84d2-2a40cdfee186 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.274357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.275001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.275263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.276065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.276971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.278941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.312846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 844/1694] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:28 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:28.314008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.329201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:28.357096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 851/1695] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:28 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:28.359833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.477625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d3a81-98de-45ef-8a0f-085d1764d589 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.589385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8542a-1734-4fbe-8233-629856f9e9b5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.600562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.601269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.601622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.603774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.608532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.610754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.639214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 852/1696] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:28.639214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.657326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.660396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.660689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.660967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.669084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af55b7d-6c85-4d18-a615-58bffe61a310 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.681302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.682327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.682826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.683293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.684219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.689864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.712730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:28.752868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 853/1697] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:28 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:28.760064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.784138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d4fce-afc9-4d1a-a4da-1eb1d4f45bfb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.798177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.800980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.802101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.802608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.804703 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.811637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.835551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 854/1698] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:28.837101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.852556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa6dfc-ce1e-48bb-9d77-a91057ef06fa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.861021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.864242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.864847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.865326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.866373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.868256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.886797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 855/1699] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:28.888113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:28.890609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:28.897527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:28.902622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10b067-0efc-48e9-a6d3-ee9821cad236 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:28.918398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f961330-890e-4d85-872b-e5f5772c5e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:28.930339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961330-890e-4d85-872b-e5f5772c5e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:28.930730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961330-890e-4d85-872b-e5f5772c5e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:28.930963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961330-890e-4d85-872b-e5f5772c5e18 admin admin] PATH_INFO: `/v3/projects/641fe924d4684aecbdcea861184019fe/users/6a7c30c4b8a948e99694fee159f15aa3/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:28.970291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 845/1700] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:28.974395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:29.013485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038ab9-402e-4702-9f97-3ebee0a16f91 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:29.016621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f961330-890e-4d85-872b-e5f5772c5e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=641fe924d4684aecbdcea861184019fe, user_id=6a7c30c4b8a948e99694fee159f15aa3, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:29.025180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f961330-890e-4d85-872b-e5f5772c5e18 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:29.028004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:29.028922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.029879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.030902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:29.033983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:29.036971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:29.073041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 856/1701] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:28 2022] PUT /identity/v3/projects/641fe924d4684aecbdcea861184019fe/users/6a7c30c4b8a948e99694fee159f15aa3/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:29.083853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.084238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.084779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 846/1702] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:29.086156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:29.087737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:29.132381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f30e3-a27c-493e-ac9b-8e49b47242ce admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:29.146810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:29.148776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:29.197686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:29.354043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66027-3245-4f4c-8e07-509b240f830f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:29.359343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66027-3245-4f4c-8e07-509b240f830f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.359804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66027-3245-4f4c-8e07-509b240f830f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.360257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66027-3245-4f4c-8e07-509b240f830f admin admin] PATH_INFO: `/v3/projects/a3b57a0634a94f71bfa506115daa48a0/users/3020e6de1e7042e3b90d5b33eac99301/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:29.405980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e66027-3245-4f4c-8e07-509b240f830f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b57a0634a94f71bfa506115daa48a0, user_id=3020e6de1e7042e3b90d5b33eac99301, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:29.415600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 857/1703] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:29.417683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e66027-3245-4f4c-8e07-509b240f830f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:29.418720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:29.448365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 847/1704] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:29 2022] PUT /identity/v3/projects/a3b57a0634a94f71bfa506115daa48a0/users/3020e6de1e7042e3b90d5b33eac99301/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:29.459035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.459322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.462223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:29.530401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] MFA Rules not processed for user `3020e6de1e7042e3b90d5b33eac99301`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:29.533137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:29.579351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dbad7-9efb-4a20-a81a-e2db2c290478 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:29.679120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 848/1705] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:29.683508 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:29.722162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:29.725930 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.726380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.727471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:29.734186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:29.738919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:29.890293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59a757-8afc-401b-ab53-55fb53133ea5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:29.901306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c882e6-ac9d-4bb2-ad11-aa08a6787865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:29.903095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c882e6-ac9d-4bb2-ad11-aa08a6787865 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.903397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c882e6-ac9d-4bb2-ad11-aa08a6787865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.906079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c882e6-ac9d-4bb2-ad11-aa08a6787865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:29.906875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c882e6-ac9d-4bb2-ad11-aa08a6787865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:29.913336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c882e6-ac9d-4bb2-ad11-aa08a6787865 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:29.926582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:29.928367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:29.960172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 858/1706] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:29.960944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:29.969859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:29.982292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 849/1707] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:29 2022] POST /identity/v3/projects => generated 369 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:29.983145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:29.985348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:29.987342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:30.046806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:30.051011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:30.074841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:30.130288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e0556-9eff-496e-b297-4a621d7c5bd1 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:30.201052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 850/1708] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:31:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:30.203114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:30.321635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:30.322414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:30.323170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:30.323170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:30.453463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84c57-8f84-4524-87fa-04e23c191ee0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:30.465861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:30.466582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:30.467988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:30.467988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:30.470126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:30.479129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:30.520216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:30.523574 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:30.546106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 851/1709] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:30 2022] POST /identity/v3/users => generated 291 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:30.550531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:30.553064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:30.575325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7a6d5-8e8e-405d-8629-9d43279ee7fa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:30.576090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 859/1710] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:30.582510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:30.588544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:30.590645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:30.594377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:30.596312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:30.597556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] PATH_INFO: `/v3/users/2b7e594fef204cf48bde8b68b7c6f2f8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:30.602491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7e594fef204cf48bde8b68b7c6f2f8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:30.607732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:30.704476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:30.743927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7e594fef204cf48bde8b68b7c6f2f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:30.761536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7e594fef204cf48bde8b68b7c6f2f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:30.762495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7e594fef204cf48bde8b68b7c6f2f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:30.763591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7e594fef204cf48bde8b68b7c6f2f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:30.787034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab8471-616d-4141-9b8b-2a7e014006c5 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:30.797054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7e594fef204cf48bde8b68b7c6f2f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:30.803097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:30.804054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:30.807063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:30.808164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:30.813982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:30.834625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 852/1711] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:30 2022] DELETE /identity/v3/users/2b7e594fef204cf48bde8b68b7c6f2f8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:30.837232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:30.840807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:30.844435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:30.848554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62b34-eb03-49c7-9ce8-1e06a88700d4 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:30.873631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 860/1712] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:31:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:31:30.882006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:30.882517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:30.882866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:30.950759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:30.955499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:31.076375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:31.173750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195da8d-580e-493a-9dc1-06caff73f8df admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:31.205254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:31.205836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:31.206130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:31.206942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:31.209092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:31.212254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:31.246349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 853/1713] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:31.251637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:31.278329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 861/1714] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:31.291605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7995136-e06c-415f-a6d2-f1fb1f461473 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:31.309872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:31.313334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:31.313803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:31.314245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] PATH_INFO: `/v3/projects/6f7e0645a90e4d0d8cc4f8b897f96385/users/7257a288b4ad4f90b0a9e26d3c0c9e25/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:31.394263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7e0645a90e4d0d8cc4f8b897f96385, user_id=7257a288b4ad4f90b0a9e26d3c0c9e25, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:31.402937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:31.448344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 854/1715] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:31 2022] PUT /identity/v3/projects/6f7e0645a90e4d0d8cc4f8b897f96385/users/7257a288b4ad4f90b0a9e26d3c0c9e25/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:31.452655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:31.464295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eb502-e3dc-4097-978a-e2150a9f791e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:31.642147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:31.644872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:31.755945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:31.756729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:31.757083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:31.757364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:31.767706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:31.771627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:31.810006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c823-ed40-4078-bed1-57da7b595ace admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:31.834188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 862/1716] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:31.835844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:31.836502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3be98-4c07-40fd-b6d3-cd6df84c8341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:31.838672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3be98-4c07-40fd-b6d3-cd6df84c8341 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:31.840563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3be98-4c07-40fd-b6d3-cd6df84c8341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:31.841680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3be98-4c07-40fd-b6d3-cd6df84c8341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:31.879687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458d5cd-9951-4149-af88-b767653e1daf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:31.907636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42824b-567f-41f8-ade7-58b97fbc9bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:31.908704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42824b-567f-41f8-ade7-58b97fbc9bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:31.908988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42824b-567f-41f8-ade7-58b97fbc9bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:31.909265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42824b-567f-41f8-ade7-58b97fbc9bc0 admin admin] PATH_INFO: `/v3/projects/6f7e0645a90e4d0d8cc4f8b897f96385/users/7257a288b4ad4f90b0a9e26d3c0c9e25/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:31.918651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42824b-567f-41f8-ade7-58b97fbc9bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7e0645a90e4d0d8cc4f8b897f96385, user_id=7257a288b4ad4f90b0a9e26d3c0c9e25, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:31.922019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42824b-567f-41f8-ade7-58b97fbc9bc0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:31.923279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3be98-4c07-40fd-b6d3-cd6df84c8341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:31.930555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3be98-4c07-40fd-b6d3-cd6df84c8341 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:32.004226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 863/1717] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:31 2022] PUT /identity/v3/projects/6f7e0645a90e4d0d8cc4f8b897f96385/users/7257a288b4ad4f90b0a9e26d3c0c9e25/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:32.006527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:32.006822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:32.007117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:32.032106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 855/1718] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:32.122148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] MFA Rules not processed for user `7257a288b4ad4f90b0a9e26d3c0c9e25`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:32.123808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:32.160233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:32.161051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:32.161795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:32.213227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] MFA Rules not processed for user `14da69798e344e70ba64bf8187523e45`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:32.215218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:32.293298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:32.401466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 864/1719] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:32.415062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:32.417343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:32.472730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 856/1720] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:32.612528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:32.719590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cb19f-6c1b-46a4-96a7-c3467a7c2a21 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:32.742524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01903d3-04af-46d3-ad67-7e88c92de374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:32.744677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01903d3-04af-46d3-ad67-7e88c92de374 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:32.746677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01903d3-04af-46d3-ad67-7e88c92de374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:32.747023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01903d3-04af-46d3-ad67-7e88c92de374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:32.806579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01903d3-04af-46d3-ad67-7e88c92de374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:32.808236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01903d3-04af-46d3-ad67-7e88c92de374 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:32.864896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 865/1721] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:32.991901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:33.009291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2f05-41e6-4bb4-a14a-b1cfa618bca7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:33.028432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:33.034154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:33.034154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:33.034154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:33.052164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:33.250884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:33.252676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:33.269635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 857/1722] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:34.624523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ebdbc-1192-4d94-bfb8-b331ec7b8943 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:34.625512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ebdbc-1192-4d94-bfb8-b331ec7b8943 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:34.625972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ebdbc-1192-4d94-bfb8-b331ec7b8943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:34.646347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:34.648738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:34.726887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a54ebdbc-1192-4d94-bfb8-b331ec7b8943 None None] MFA Rules not processed for user `2d3b91bfd0304cc793e3ad52c9b9e865`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:34.728305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a54ebdbc-1192-4d94-bfb8-b331ec7b8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:34.933948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f833d-e643-4e78-8f34-f5963c206db3 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:34.963059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:34.964481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:34.964915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:34.965432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] PATH_INFO: `/v3/users/49468ccc681c4d7293a2c18d6535a1ad` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:34.977989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=49468ccc681c4d7293a2c18d6535a1ad)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:34.984670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:35.045517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 866/1723] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:35.052829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:35.053166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:35.053398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:35.110885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] MFA Rules not processed for user `2d3b91bfd0304cc793e3ad52c9b9e865`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:35.112299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:35.161512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 867/1724] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:35.177745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49468ccc681c4d7293a2c18d6535a1ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:35.178087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:35.179400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:35.194874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49468ccc681c4d7293a2c18d6535a1ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:35.198176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49468ccc681c4d7293a2c18d6535a1ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:35.199313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49468ccc681c4d7293a2c18d6535a1ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:35.265558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:35.282188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-0ccfc365-8600-470c-85d2-6aebfaa5df0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49468ccc681c4d7293a2c18d6535a1ad'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:35.358838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 858/1725] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:34 2022] DELETE /identity/v3/users/49468ccc681c4d7293a2c18d6535a1ad => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:35.365062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:35.368477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:35.373649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:35.447718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:35.449115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:35.517991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128583d8-2f0a-4ad3-90cc-6f672ba882bc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:35.678931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 859/1726] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:35.714243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:35.729668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:35.730639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:35.731097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:35.731543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:35.734029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:35.759101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:35.766488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:35.845854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:35.848992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:35.876723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 868/1727] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:36.029541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6e025-81db-42e5-89bd-e0765e560516 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:36.035085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e098d-f64e-4739-99f0-8f0a50e43bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:36.035796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e098d-f64e-4739-99f0-8f0a50e43bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.036081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e098d-f64e-4739-99f0-8f0a50e43bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.036457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e098d-f64e-4739-99f0-8f0a50e43bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.049856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e098d-f64e-4739-99f0-8f0a50e43bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:36.051499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e098d-f64e-4739-99f0-8f0a50e43bb6 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:36.064197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 860/1728] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:36.335721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:36.337817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:36.368908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40c62-46c0-4ed9-b437-b16466bb2701 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:36.381960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:36.383018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.383398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.383769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] PATH_INFO: `/v3/projects/6c7967c1e3ad47cc9e2fbd3de2c673b5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.385166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7967c1e3ad47cc9e2fbd3de2c673b5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:36.387221 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:36.501331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7967c1e3ad47cc9e2fbd3de2c673b5', 'request_id': 'req-532af88b-2904-466e-8033-cca81634e734'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.514857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7967c1e3ad47cc9e2fbd3de2c673b5', 'request_id': 'req-532af88b-2904-466e-8033-cca81634e734'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.524105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7967c1e3ad47cc9e2fbd3de2c673b5', 'request_id': 'req-532af88b-2904-466e-8033-cca81634e734'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.536920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] The token cache is being invalidate because project 6c7967c1e3ad47cc9e2fbd3de2c673b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:36.537237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.537910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.539489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 869/1729] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:36 2022] DELETE /identity/v3/projects/6c7967c1e3ad47cc9e2fbd3de2c673b5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:36.545238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191b39c-5a68-4386-8114-b2bd2b3b3ba0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.545489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191b39c-5a68-4386-8114-b2bd2b3b3ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.545692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191b39c-5a68-4386-8114-b2bd2b3b3ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.566070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2191b39c-5a68-4386-8114-b2bd2b3b3ba0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:36.567293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2191b39c-5a68-4386-8114-b2bd2b3b3ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:36.649920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:36.651778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:36.677975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 861/1730] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:36.687053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.688067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.688534 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.713331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:36.715107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:36.738020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 862/1731] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:36.749692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:36.797453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532af88b-2904-466e-8033-cca81634e734 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:36.813800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0cb37-2cce-4054-866d-f54b0b80a962 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:36.830737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:36.830936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:36.831624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.831734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.831902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.832096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.832649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.833751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:36.834207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:36.837201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:36.841400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] PATH_INFO: `/v3/projects/f6ab199409614959939e8c00baa40d08` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.849847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ab199409614959939e8c00baa40d08)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:36.862642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:36.862815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 863/1732] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:36.863922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:36.886177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff1bce-642e-470a-882f-d190549ea87d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:36.903674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:36.910679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d8d2b-f006-4a63-b38d-9f26cdd1e253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:36.915726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d8d2b-f006-4a63-b38d-9f26cdd1e253 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.916040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d8d2b-f006-4a63-b38d-9f26cdd1e253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:36.916773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d8d2b-f006-4a63-b38d-9f26cdd1e253 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:36.918156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d8d2b-f006-4a63-b38d-9f26cdd1e253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:36.922473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d8d2b-f006-4a63-b38d-9f26cdd1e253 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:36.929468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:36.962637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ab199409614959939e8c00baa40d08', 'request_id': 'req-6128ac42-1f89-4548-9ba8-a357d6cb0076'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.982484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ab199409614959939e8c00baa40d08', 'request_id': 'req-6128ac42-1f89-4548-9ba8-a357d6cb0076'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:36.999624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 864/1733] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:36 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:36.999624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c632d74-6c6c-48a5-9869-10b42edbed6c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:36.999624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c632d74-6c6c-48a5-9869-10b42edbed6c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.002810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c632d74-6c6c-48a5-9869-10b42edbed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.019021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ab199409614959939e8c00baa40d08', 'request_id': 'req-6128ac42-1f89-4548-9ba8-a357d6cb0076'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:37.059817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] The token cache is being invalidate because project f6ab199409614959939e8c00baa40d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:37.060421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:37.061653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:37.068171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 870/1734] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:36 2022] DELETE /identity/v3/projects/f6ab199409614959939e8c00baa40d08 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:37.069963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:37.071626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:37.112176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7c632d74-6c6c-48a5-9869-10b42edbed6c None None] MFA Rules not processed for user `6a7c30c4b8a948e99694fee159f15aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:37.113655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7c632d74-6c6c-48a5-9869-10b42edbed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:37.270330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128ac42-1f89-4548-9ba8-a357d6cb0076 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:37.289995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 865/1735] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:37.293968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.294430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.294884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.377700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:37.382012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:37.387557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf58bc2-0c46-4959-8bd2-918fc1cdcc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:37.388992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf58bc2-0c46-4959-8bd2-918fc1cdcc9b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.389329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf58bc2-0c46-4959-8bd2-918fc1cdcc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.389638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf58bc2-0c46-4959-8bd2-918fc1cdcc9b admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.393920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf58bc2-0c46-4959-8bd2-918fc1cdcc9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:37.401065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf58bc2-0c46-4959-8bd2-918fc1cdcc9b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:37.412909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 866/1736] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:37.413781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:37.438927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:37.473563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 871/1737] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/users => generated 286 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:37.476155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027815ec-a5f0-4f05-b863-bc25a4aebc18 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.476456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027815ec-a5f0-4f05-b863-bc25a4aebc18 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.479596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027815ec-a5f0-4f05-b863-bc25a4aebc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.489066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229c282-a4cd-4bd8-bb7b-475b6684f729 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:37.507303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:37.507980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.508261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.508533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.509230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:37.511170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:37.572143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-027815ec-a5f0-4f05-b863-bc25a4aebc18 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:37.576913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-027815ec-a5f0-4f05-b863-bc25a4aebc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:37.610673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 867/1738] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:37.610673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:37.610944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 872/1739] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:37.612237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.612524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.612797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.662435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] MFA Rules not processed for user `2d3b91bfd0304cc793e3ad52c9b9e865`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:37.665971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:37.758435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:37.771246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f3615-26a2-4b8a-aae5-7b190f0016e9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:37.786933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:37.787754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.788044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.788353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.789147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:37.791608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:37.807760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 868/1740] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:37.808746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:37.813112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 873/1741] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:37.814048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:37.850478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a85e4-195a-4da3-9f76-0ce1e2307d69 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:37.852881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84d17-a6a8-4b4c-b999-685a3fafa80e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:37.864160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:37.864844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.865168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.865389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.866145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:37.866447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:37.868783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:37.878472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:37.879727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:37.879727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:37.879727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:37.880379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:37.882839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:37.914884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 869/1742] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:37.914884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:37.951682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 874/1743] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:37.958936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:38.009925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eb2e5-0306-4e76-b106-a89241320de0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.021633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0e1f4-8458-4642-8b55-8034d23a5119 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.032649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:38.033203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.033579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.033968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.034861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:38.037196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:38.048241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cc7b8-7f6c-4854-8b82-6245b5897204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:38.049074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc7b8-7f6c-4854-8b82-6245b5897204 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.049670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc7b8-7f6c-4854-8b82-6245b5897204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.050036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cc7b8-7f6c-4854-8b82-6245b5897204 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.051104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc7b8-7f6c-4854-8b82-6245b5897204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:38.085124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cc7b8-7f6c-4854-8b82-6245b5897204 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:38.119259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 870/1744] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:38.127815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:38.184029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 875/1745] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:37 2022] POST /identity/v3/projects => generated 385 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:38.185999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.186369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.186716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.192108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad9cce-2af2-4828-b221-020c8350c947 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.261736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:38.265121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:38.269222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:38.415863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:38.418179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.420658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.421295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.424709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:38.427290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:38.478135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 871/1746] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:38.479219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:38.522663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 876/1747] 10.222.0.58 () {58 vars in 1015 bytes} [Thu May 19 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:38.523709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:38.524153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c57cdf-e9cc-4cb8-87c9-1b2f4ef09357 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.552408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:38.554519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.559936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.561706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] PATH_INFO: `/v3/projects/c7c73b53e1e140b281f590dd5f33e3a4/users/0a56dae1b5c547d3af68ade6ab7cd467/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.580196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d5d84-af0e-4386-94de-145b35c2cc3a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.597314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:38.603124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.606342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.606686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.607575 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:38.624140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:38.648697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c73b53e1e140b281f590dd5f33e3a4, user_id=0a56dae1b5c547d3af68ade6ab7cd467, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:38.667161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:38.732354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 877/1748] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:38 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:38.735536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:38.737467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 872/1749] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:38 2022] PUT /identity/v3/projects/c7c73b53e1e140b281f590dd5f33e3a4/users/0a56dae1b5c547d3af68ade6ab7cd467/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:38.739028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:38.755439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:38.792953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aaaeb-d176-4515-ab2c-e58424ea8c1f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.909705 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac212098-9fcd-4992-acd4-ec085a1353ca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:38.914839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:38.915670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:38.916651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:38.917341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:38.927696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:38.930077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:38.944477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 873/1750] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:38.946045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:39.082550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:39.085128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:39.085445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:39.085724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] PATH_INFO: `/v3/projects/70fbb1ed0fd3462db16a7fbd4571722b/users/80ebbc6807f1425380c56322b09d17f7/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:39.140120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935f351-984d-444a-8d5a-fe2a7b9407b5 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:39.165190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade9705-de84-4f59-a936-4cf6c1b8c8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:39.167738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fbb1ed0fd3462db16a7fbd4571722b, user_id=80ebbc6807f1425380c56322b09d17f7, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:39.168149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9705-de84-4f59-a936-4cf6c1b8c8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:39.168580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9705-de84-4f59-a936-4cf6c1b8c8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:39.169784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade9705-de84-4f59-a936-4cf6c1b8c8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:39.172249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9705-de84-4f59-a936-4cf6c1b8c8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:39.173723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:39.180182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade9705-de84-4f59-a936-4cf6c1b8c8ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:39.211218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 874/1751] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:39.212811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 878/1752] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:38 2022] PUT /identity/v3/projects/70fbb1ed0fd3462db16a7fbd4571722b/users/80ebbc6807f1425380c56322b09d17f7/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:39.213810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:39.215311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:39.215738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:39.216202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:39.219046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:39.296902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:39.320203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] MFA Rules not processed for user `0a56dae1b5c547d3af68ade6ab7cd467`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:39.321545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:39.433809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:39.609863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 875/1753] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:39.611553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:39.619557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97969fd2-9831-4bc5-bfe6-924479431cec admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:39.649137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:39.661567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:39.661706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7399661-dc5a-4f46-9f10-923fc3e860d2 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:39.661868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:39.665551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] PATH_INFO: `/v3/users/3020e6de1e7042e3b90d5b33eac99301` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:39.667495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3020e6de1e7042e3b90d5b33eac99301)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:39.690267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:39.690966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:39.691239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:39.691519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:39.692263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:39.693916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:39.694954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:39.759068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 876/1754] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:39.763288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:39.803429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb179a-0f5a-40e6-a41f-f98434c3d475 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:39.822547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:39.824459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:39.832437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:39.840231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:39.841601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:39.845378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:39.887117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 877/1755] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:39.888918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:39.895402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:39.905444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3020e6de1e7042e3b90d5b33eac99301'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:39.964210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3020e6de1e7042e3b90d5b33eac99301'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:39.968320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3020e6de1e7042e3b90d5b33eac99301'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:39.971862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3020e6de1e7042e3b90d5b33eac99301'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:40.010794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3020e6de1e7042e3b90d5b33eac99301'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:40.044965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 879/1756] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:39 2022] DELETE /identity/v3/users/3020e6de1e7042e3b90d5b33eac99301 => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:40.046079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:40.065247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:40.183330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426e431-2a5a-436a-8ae3-bd09e7ec5760 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:40.379105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ec24e-d92f-4991-8499-9b7b0270e3a1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:40.452952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:40.460398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:40.460598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:40.466155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] PATH_INFO: `/v3/projects/8493133d405443efb4451fe7415ab675/users/21611090b48245d9930a710f1aea630c/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:40.510511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105e346-5419-4cf2-a340-46923608860f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:40.511439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e346-5419-4cf2-a340-46923608860f service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:40.516164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e346-5419-4cf2-a340-46923608860f service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:40.518954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e346-5419-4cf2-a340-46923608860f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:40.542346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8105e346-5419-4cf2-a340-46923608860f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:40.598408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8493133d405443efb4451fe7415ab675, user_id=21611090b48245d9930a710f1aea630c, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:40.646757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:40.652317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:40.723811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e346-5419-4cf2-a340-46923608860f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:40.729047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e346-5419-4cf2-a340-46923608860f service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:40.736764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 880/1757] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:40 2022] PUT /identity/v3/projects/8493133d405443efb4451fe7415ab675/users/21611090b48245d9930a710f1aea630c/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:40.738126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:40.748748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:40.769814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 878/1758] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 882 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:40.771289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:40.771586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:40.772442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:40.877445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:40.884427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:40.891219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:41.012686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab373f-46c8-4770-933a-486e01fbe7dd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:41.027950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b38d5-ad77-49c2-85fe-cf314793700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:41.028972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b38d5-ad77-49c2-85fe-cf314793700f admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:41.029340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b38d5-ad77-49c2-85fe-cf314793700f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:41.030053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b38d5-ad77-49c2-85fe-cf314793700f admin admin] PATH_INFO: `/v3/projects/70fbb1ed0fd3462db16a7fbd4571722b/users/80ebbc6807f1425380c56322b09d17f7/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:41.036007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b38d5-ad77-49c2-85fe-cf314793700f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fbb1ed0fd3462db16a7fbd4571722b, user_id=80ebbc6807f1425380c56322b09d17f7, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:41.039236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b38d5-ad77-49c2-85fe-cf314793700f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:41.102045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 881/1759] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:40 2022] PUT /identity/v3/projects/70fbb1ed0fd3462db16a7fbd4571722b/users/80ebbc6807f1425380c56322b09d17f7/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:41.103680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:41.103972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:41.104274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:41.146270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 879/1760] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:41.221554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] MFA Rules not processed for user `21611090b48245d9930a710f1aea630c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:41.225427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:41.314187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:41.315897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:41.604212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 882/1761] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:41.606610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:41.651285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860714df-9af1-47be-80c0-0eab3b7e4103 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:41.905063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:41.905812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:41.906608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:41.912176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:41.923467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0301ad-2ffa-4ecf-8226-44d7b907e001 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:41.937961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:41.942854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:41.952557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:41.958126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:41.958531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:41.958817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:41.959583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:41.965620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:41.977551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 880/1762] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:42.057480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 883/1763] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:41 2022] POST /identity/v3/projects => generated 385 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:42.058758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:42.074906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0ff6-498a-4a12-be33-3c0ef3e4bd30 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:42.171017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:42.182911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:42.204396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:42.299440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:42.301762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.302323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.303192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.305347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:42.308422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:42.314328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:42.358926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 881/1764] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:42 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:31:42.363184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:42.398764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214662c-82c9-4483-b6e7-8c8c5d228135 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:42.411585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d07ea-1169-4105-acd7-0d849de9441e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:42.414845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:42.415480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.415742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.416061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.417049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:42.418752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:42.431053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:42.431950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.432234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.432507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] PATH_INFO: `/v3/users/e2e539dc5ad6438dbe988e5ccd4c89e3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.435286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e539dc5ad6438dbe988e5ccd4c89e3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:42.437803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:42.448583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 882/1765] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:42.449575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:42.483482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a9810-93c9-4885-a926-a23e5af1d8be admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:42.500643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a80be7-c9a2-4d0e-9362-b6351449b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:42.501350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80be7-c9a2-4d0e-9362-b6351449b42b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.501642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80be7-c9a2-4d0e-9362-b6351449b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.501914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a80be7-c9a2-4d0e-9362-b6351449b42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.502660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80be7-c9a2-4d0e-9362-b6351449b42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:42.515976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a80be7-c9a2-4d0e-9362-b6351449b42b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:42.562802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 883/1766] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:42.564421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15470514-3534-4284-a8a7-e7add560790d None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.564709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15470514-3534-4284-a8a7-e7add560790d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.564984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15470514-3534-4284-a8a7-e7add560790d None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.568270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 884/1767] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:31:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:42.569067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15470514-3534-4284-a8a7-e7add560790d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:42.582928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e539dc5ad6438dbe988e5ccd4c89e3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:42.620740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15470514-3534-4284-a8a7-e7add560790d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:42.627652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e539dc5ad6438dbe988e5ccd4c89e3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:42.635476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e539dc5ad6438dbe988e5ccd4c89e3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:42.636736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e539dc5ad6438dbe988e5ccd4c89e3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:42.640200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d179e13-452c-41ae-860f-7bd48a29c181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:42.642362 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179e13-452c-41ae-860f-7bd48a29c181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.642645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179e13-452c-41ae-860f-7bd48a29c181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.642925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d179e13-452c-41ae-860f-7bd48a29c181 admin admin] PATH_INFO: `/v3/projects/4bd72c953cfc4fd9844500f1d5f7795d/users/8354bbff30164fc3a9778f9bb2828ca0/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.663818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9949ebc8-2a1b-44a4-8cdb-06e9c8af8973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e539dc5ad6438dbe988e5ccd4c89e3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:42.689459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d179e13-452c-41ae-860f-7bd48a29c181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd72c953cfc4fd9844500f1d5f7795d, user_id=8354bbff30164fc3a9778f9bb2828ca0, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:42.697571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d179e13-452c-41ae-860f-7bd48a29c181 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:42.726125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 884/1768] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:42 2022] DELETE /identity/v3/users/e2e539dc5ad6438dbe988e5ccd4c89e3 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:42.728906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.730267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.730918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.746523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 885/1769] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:42 2022] PUT /identity/v3/projects/4bd72c953cfc4fd9844500f1d5f7795d/users/8354bbff30164fc3a9778f9bb2828ca0/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:42.752045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767457-7c62-457f-8f8e-c7405becc08d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:42.752341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767457-7c62-457f-8f8e-c7405becc08d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:42.756409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a767457-7c62-457f-8f8e-c7405becc08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:42.796740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:42.799713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:42.843289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7a767457-7c62-457f-8f8e-c7405becc08d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:42.844908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a767457-7c62-457f-8f8e-c7405becc08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:43.024081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 886/1770] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:43.025790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.026081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.026359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:43.113032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 885/1771] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:43.113242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] MFA Rules not processed for user `8354bbff30164fc3a9778f9bb2828ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:43.119166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:43.230464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:43.232910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:43.325452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 887/1772] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:43.330618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:43.334303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:43.374517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481626b-a474-4c0c-8544-03c969ffc81c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:43.382120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:43.382942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.383347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.383782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:43.385003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:43.389572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:43.394923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:43.417273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:43.468288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a07b4-969f-4d20-b3dc-6ad56b5a92b8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:43.487757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:43.488684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.493324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.493734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:43.505983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 888/1773] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:43 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:43.510888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:43.516224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:43.517958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:43.554328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 886/1774] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:43.588414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:43.590157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:43.637666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8d616-d6b0-4f70-9884-326b4b3a31db service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:43.715117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa22068-5363-42d1-975e-40244e030ae4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:43.739899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:43.749316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.750116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.750598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:43.752205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:43.754860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:43.792924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 889/1775] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:43 2022] POST /identity/v3/users => generated 300 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:43.796661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:43.816953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:43.817695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.817980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.818255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:43.828047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1df7f-6960-445b-b262-471f5ecb90ef admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:43.830978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:43.832878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:43.845395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:43.846116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.846401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.846673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:43.847418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:43.849189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:43.850869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 887/1776] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:43.893903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 890/1777] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:43.898162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:43.916866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60ca0-4ad5-40e1-ae9d-9ad194d0da36 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:43.940070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:43.942324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:43.945036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:43.945565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] PATH_INFO: `/v3/projects/2ce701bafcc64ed4915adf8b7337b4b0/users/a20d3c1ee68b434788bd1d8e4c1e4d85/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:44.001151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce701bafcc64ed4915adf8b7337b4b0, user_id=a20d3c1ee68b434788bd1d8e4c1e4d85, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:44.008458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:44.044756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 888/1778] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:43 2022] PUT /identity/v3/projects/2ce701bafcc64ed4915adf8b7337b4b0/users/a20d3c1ee68b434788bd1d8e4c1e4d85/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:44.046165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:44.058198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9462bc1-4e2f-46e2-9eb8-0900c92fd77e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:44.230017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:44.232455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:44.280659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:44.286392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:44.286690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:44.287612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:44.287789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:44.308393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:44.373761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 891/1779] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:31:44.376173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:44.400048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc50a6f-4110-4b07-bab8-91bac6967ef6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:44.407160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c83d-97fc-4941-a92e-3b04da44b47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:44.408407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c83d-97fc-4941-a92e-3b04da44b47f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:44.409389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c83d-97fc-4941-a92e-3b04da44b47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:44.410334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c83d-97fc-4941-a92e-3b04da44b47f admin admin] PATH_INFO: `/v3/projects/2ce701bafcc64ed4915adf8b7337b4b0/users/a20d3c1ee68b434788bd1d8e4c1e4d85/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:44.422075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322c83d-97fc-4941-a92e-3b04da44b47f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce701bafcc64ed4915adf8b7337b4b0, user_id=a20d3c1ee68b434788bd1d8e4c1e4d85, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:44.425195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322c83d-97fc-4941-a92e-3b04da44b47f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:44.475654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 892/1780] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:44 2022] PUT /identity/v3/projects/2ce701bafcc64ed4915adf8b7337b4b0/users/a20d3c1ee68b434788bd1d8e4c1e4d85/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:44.477958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:44.478318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:44.478603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:44.499887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9950d-d58b-4abb-b8b9-8a93f086b0b8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:44.520253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] MFA Rules not processed for user `a20d3c1ee68b434788bd1d8e4c1e4d85`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:44.521554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:44.620371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:44.650889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:44.680836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:44.683155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:44.683443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:44.683733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] PATH_INFO: `/v3/projects/a3b57a0634a94f71bfa506115daa48a0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:44.688022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b57a0634a94f71bfa506115daa48a0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:44.696550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:44.722338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 893/1781] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:44.730445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:44.731910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:44.820313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b57a0634a94f71bfa506115daa48a0', 'request_id': 'req-278eb892-24c7-45fe-b077-02bdad659f67'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:44.837809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b57a0634a94f71bfa506115daa48a0', 'request_id': 'req-278eb892-24c7-45fe-b077-02bdad659f67'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:44.851276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b57a0634a94f71bfa506115daa48a0', 'request_id': 'req-278eb892-24c7-45fe-b077-02bdad659f67'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:44.877186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] The token cache is being invalidate because project a3b57a0634a94f71bfa506115daa48a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:44.878101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:44.879163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:44.880095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0ff08-a710-402d-9d0a-3689c260040a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:44.885998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 889/1782] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:44 2022] DELETE /identity/v3/projects/a3b57a0634a94f71bfa506115daa48a0 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:44.892335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:44.894866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:45.034300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:45.035825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:45.036252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:45.036668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:45.040444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:45.121826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eb892-24c7-45fe-b077-02bdad659f67 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:45.137802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:45.141966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:45.142356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:45.143134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] PATH_INFO: `/v3/users/7257a288b4ad4f90b0a9e26d3c0c9e25` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:45.148429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7257a288b4ad4f90b0a9e26d3c0c9e25)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:45.153500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:45.240831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:45.243105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:45.266403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 894/1783] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:45.267564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:45.269057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:45.286220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebd25b-9f2a-49fb-a314-f991ffc61773 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:45.292618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:45.293533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:45.293980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:45.294265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] PATH_INFO: `/v3/users/0a56dae1b5c547d3af68ade6ab7cd467` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:45.297091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a56dae1b5c547d3af68ade6ab7cd467)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:45.297957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7257a288b4ad4f90b0a9e26d3c0c9e25'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.299165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:45.335072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7257a288b4ad4f90b0a9e26d3c0c9e25'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.336213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7257a288b4ad4f90b0a9e26d3c0c9e25'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.337631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7257a288b4ad4f90b0a9e26d3c0c9e25'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.351499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7257a288b4ad4f90b0a9e26d3c0c9e25'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.385367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:45.394831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 890/1784] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:44 2022] DELETE /identity/v3/users/7257a288b4ad4f90b0a9e26d3c0c9e25 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:45.395932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:45.400127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:45.410648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:45.416535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a56dae1b5c547d3af68ade6ab7cd467'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.437298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a56dae1b5c547d3af68ade6ab7cd467'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.437976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a56dae1b5c547d3af68ade6ab7cd467'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.440188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a56dae1b5c547d3af68ade6ab7cd467'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.467533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0e0674a-1971-4c34-be18-dd4022171702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a56dae1b5c547d3af68ade6ab7cd467'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:45.495857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 895/1785] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:45 2022] DELETE /identity/v3/users/0a56dae1b5c547d3af68ade6ab7cd467 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:45.497335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:45.501967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:45.502297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:45.538544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:45.539797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:45.608105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6eaee-837c-49be-b7ce-eda15a442e36 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:45.687839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 896/1786] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:45.806565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:45.808458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:45.883515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:45.888758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:45.889058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:45.889347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] PATH_INFO: `/v3/projects/6a7ac23cb05647a4b5cbc1524c355228` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:45.895105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7ac23cb05647a4b5cbc1524c355228)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:45.902289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:45.954485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc542a1-b64f-4eb6-99bf-22bc40f9e59b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:45.962160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:45.962975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:45.963310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:45.963631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:45.987154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:45.988955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:46.016546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 897/1787] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:46.017232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7ac23cb05647a4b5cbc1524c355228', 'request_id': 'req-76eb9bae-9519-42e4-a8c3-32c4621611b4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.028864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7ac23cb05647a4b5cbc1524c355228', 'request_id': 'req-76eb9bae-9519-42e4-a8c3-32c4621611b4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.037439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7ac23cb05647a4b5cbc1524c355228', 'request_id': 'req-76eb9bae-9519-42e4-a8c3-32c4621611b4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.052478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] The token cache is being invalidate because project 6a7ac23cb05647a4b5cbc1524c355228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:46.053861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.054874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.057559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 891/1788] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:45 2022] DELETE /identity/v3/projects/6a7ac23cb05647a4b5cbc1524c355228 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:46.062051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:46.065624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:46.084659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ed20-68ae-476e-940b-9dfc39ee82ad service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:46.160579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:46.186343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:46.187265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:46.189663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:46.189952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] PATH_INFO: `/v3/users/8033e746642d4ad7bc04ae83ee469b1d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:46.191662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8033e746642d4ad7bc04ae83ee469b1d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:46.193805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:46.260002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8033e746642d4ad7bc04ae83ee469b1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.277755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8033e746642d4ad7bc04ae83ee469b1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.278583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8033e746642d4ad7bc04ae83ee469b1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.279951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8033e746642d4ad7bc04ae83ee469b1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.311433 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8033e746642d4ad7bc04ae83ee469b1d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:46.352502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 898/1789] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:46 2022] DELETE /identity/v3/users/8033e746642d4ad7bc04ae83ee469b1d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:46.432514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:46.434018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:46.490327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:46.513430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb9bae-9519-42e4-a8c3-32c4621611b4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:46.586912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:46.587510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:46.587790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:46.588064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:46.589928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:46.700189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:46.703435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14022df4-e0ae-4899-a714-bd2473273086 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:46.733506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 892/1790] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:46.779658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:46.781345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:46.951869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50ac9d-9329-4fc4-ae6a-d6ceb2065241 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:46.995689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:46.996885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:46.997325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:46.997977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] PATH_INFO: `/v3/projects/6f7e0645a90e4d0d8cc4f8b897f96385` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.002602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7e0645a90e4d0d8cc4f8b897f96385)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:47.004899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:47.139106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7e0645a90e4d0d8cc4f8b897f96385', 'request_id': 'req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:47.171634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7e0645a90e4d0d8cc4f8b897f96385', 'request_id': 'req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:47.200016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7e0645a90e4d0d8cc4f8b897f96385', 'request_id': 'req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:47.225986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:47.226490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:47.226564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.246610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] The token cache is being invalidate because project 6f7e0645a90e4d0d8cc4f8b897f96385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:47.247225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:47.249586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-559a1c58-96b8-4b6c-90e4-dca1ef2cb38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:47.253232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 899/1791] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:46 2022] DELETE /identity/v3/projects/6f7e0645a90e4d0d8cc4f8b897f96385 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:47.259224 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a638-6601-4a4c-a6ab-f6eee1be44bd None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:47.259604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a638-6601-4a4c-a6ab-f6eee1be44bd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:47.260342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a638-6601-4a4c-a6ab-f6eee1be44bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.281975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:47.320637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0396a638-6601-4a4c-a6ab-f6eee1be44bd None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:47.322060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0396a638-6601-4a4c-a6ab-f6eee1be44bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:47.327996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] MFA Rules not processed for user `80ebbc6807f1425380c56322b09d17f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:47.337950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:47.482293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 900/1792] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:47.490166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:47.490507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:47.490734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.545918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:47.547187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:47.566037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 901/1793] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:47.568942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:47.582161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 893/1794] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:47.588547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaaf28-8ace-4d8c-9f70-5df8da84b988 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:47.595487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:47.596915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:47.607070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:47.608280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:47.608280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:47.608430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.609249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:47.609459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:47.611120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:47.649499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 902/1795] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:47.655880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:47.675849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749d1bf-e9a8-42f1-ae18-5b70cb2d888e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:47.690939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:47.691634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:47.691916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:47.692193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.692948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:47.695228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:47.736502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:47.800910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bdcec-1ac6-4b7a-b2ae-ae0014a0e42d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:47.810938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:47.811623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:47.811910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:47.812190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:47.818890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:47.946758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:47.948374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:47.964835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 894/1796] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:47.967332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:47.967503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 903/1797] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:47 2022] POST /identity/v3/projects => generated 387 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:47.969361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:47.971801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:47.997206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4c8f1-af99-4f35-856f-cdada2c49f39 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.107958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa87b40-57d0-4fe2-9658-00d88a9cb410 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.123418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.124992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.127736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.129267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.133091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.139635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.141156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.142097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.142799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.144785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] PATH_INFO: `/v3/projects/4f0120f6bbdc49a6b5989b906dc35af8` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.147009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0120f6bbdc49a6b5989b906dc35af8)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.150443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.194303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 904/1798] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:47 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:48.194303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.195805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.219463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d95d7-e688-466e-88be-0a50d33603ca admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.225459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.226532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.226878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.227154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] PATH_INFO: `/v3/projects/c7c73b53e1e140b281f590dd5f33e3a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.229178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c73b53e1e140b281f590dd5f33e3a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.231441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.246757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0120f6bbdc49a6b5989b906dc35af8', 'request_id': 'req-ef268080-4568-4a13-b0d8-db062fccf752'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.264944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0120f6bbdc49a6b5989b906dc35af8', 'request_id': 'req-ef268080-4568-4a13-b0d8-db062fccf752'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.274737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0120f6bbdc49a6b5989b906dc35af8', 'request_id': 'req-ef268080-4568-4a13-b0d8-db062fccf752'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.288727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] The token cache is being invalidate because project 4f0120f6bbdc49a6b5989b906dc35af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:48.289778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.290734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.294807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 895/1799] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:47 2022] DELETE /identity/v3/projects/4f0120f6bbdc49a6b5989b906dc35af8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:48.296069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.300860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.319949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:48.325284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c73b53e1e140b281f590dd5f33e3a4', 'request_id': 'req-23332edf-0788-40d0-8145-8bca9f7d0f7f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.339847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:48.342786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c73b53e1e140b281f590dd5f33e3a4', 'request_id': 'req-23332edf-0788-40d0-8145-8bca9f7d0f7f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.351455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c73b53e1e140b281f590dd5f33e3a4', 'request_id': 'req-23332edf-0788-40d0-8145-8bca9f7d0f7f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.373828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] The token cache is being invalidate because project c7c73b53e1e140b281f590dd5f33e3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:48.374122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.374973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-23332edf-0788-40d0-8145-8bca9f7d0f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:48.376990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 905/1800] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:48 2022] DELETE /identity/v3/projects/c7c73b53e1e140b281f590dd5f33e3a4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:48.378989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.379493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.380101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.421549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef268080-4568-4a13-b0d8-db062fccf752 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.428625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:48.431001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.516452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d91614-346a-41d3-b5ed-3fdfc6024082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.517322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d91614-346a-41d3-b5ed-3fdfc6024082 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.517963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d91614-346a-41d3-b5ed-3fdfc6024082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.518576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d91614-346a-41d3-b5ed-3fdfc6024082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.520372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d91614-346a-41d3-b5ed-3fdfc6024082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.522891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d91614-346a-41d3-b5ed-3fdfc6024082 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.538447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 896/1801] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:48.540010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894da679-91f4-42bc-93ba-d309387e75fa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.540403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894da679-91f4-42bc-93ba-d309387e75fa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.540783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894da679-91f4-42bc-93ba-d309387e75fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.549475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 906/1802] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:48.549475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.549475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.572680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fb4b0-8721-482b-b73a-6a1faf1c0e29 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.576580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-894da679-91f4-42bc-93ba-d309387e75fa None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:48.579508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-894da679-91f4-42bc-93ba-d309387e75fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.580836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4799a5-9579-4211-84c9-c1442d796185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.581653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4799a5-9579-4211-84c9-c1442d796185 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.582043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4799a5-9579-4211-84c9-c1442d796185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.582385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4799a5-9579-4211-84c9-c1442d796185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.584120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4799a5-9579-4211-84c9-c1442d796185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.585995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4799a5-9579-4211-84c9-c1442d796185 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.603176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 897/1803] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:48.604833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.605334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.605853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.606059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 907/1804] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:48.611303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.611598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.611871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.631121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:48.633199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.647476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 898/1805] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:48.648624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.660762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d6483-1a34-49aa-ad54-6282f7bc263b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.663608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:48.664871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.670206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.671676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.672043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.672953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] PATH_INFO: `/v3/projects/dfcc6ae6e0d44913a0766690d7aea0cb/users/333b149f16d84617b74176918387a270/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.675286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 908/1806] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:48.676152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.688133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136dfaf-c3af-46e7-b2b9-599e7000a7a6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.700479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcc6ae6e0d44913a0766690d7aea0cb, user_id=333b149f16d84617b74176918387a270, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.703332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.707545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.708341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.708677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.709041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.709957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:48.710358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.712044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9f43a-1cf3-478a-91bf-f91e879f8aed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.727533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 899/1807] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:48 2022] PUT /identity/v3/projects/dfcc6ae6e0d44913a0766690d7aea0cb/users/333b149f16d84617b74176918387a270/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:48.727533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.727830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 909/1808] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:48.731014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.731014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.731014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.763284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] MFA Rules not processed for user `333b149f16d84617b74176918387a270`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:48.764668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:48.821267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49511183-8372-426e-ac57-5975ef1c235e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.826248 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.827104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.827366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.827826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.828634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:48.829173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.832388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.852861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 900/1809] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:48.853857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.866887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed63808-9d61-4705-8bd2-7008b10d6b6a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:48.872176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee95d8-3b8a-494a-9b18-2f88bb95a236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:48.873044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee95d8-3b8a-494a-9b18-2f88bb95a236 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.873331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee95d8-3b8a-494a-9b18-2f88bb95a236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.873712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee95d8-3b8a-494a-9b18-2f88bb95a236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.875092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee95d8-3b8a-494a-9b18-2f88bb95a236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:48.877438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee95d8-3b8a-494a-9b18-2f88bb95a236 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:48.908326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:48.911470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 910/1810] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:48.915342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.924656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 901/1811] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:48.929299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4274b-d27b-440e-b683-125343ebbe69 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:48.929619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4274b-d27b-440e-b683-125343ebbe69 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:48.929897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4274b-d27b-440e-b683-125343ebbe69 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:48.931521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 902/1812] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:48.932223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4274b-d27b-440e-b683-125343ebbe69 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:48.942636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540eab-35c8-4c26-ae21-fb3c4290670d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.018034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:49.077530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.078238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.080274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.080530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.083887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.089753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.097301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4274b-d27b-440e-b683-125343ebbe69 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.113642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e9b07-8083-404b-b5ed-e05ce270b3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.122672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e9b07-8083-404b-b5ed-e05ce270b3da admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.124366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e9b07-8083-404b-b5ed-e05ce270b3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.125368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e9b07-8083-404b-b5ed-e05ce270b3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.143150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 911/1813] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/projects => generated 371 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:49.145642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e9b07-8083-404b-b5ed-e05ce270b3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.149199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.150565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e9b07-8083-404b-b5ed-e05ce270b3da admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.206759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 903/1814] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:48 2022] POST /identity/v3/projects => generated 385 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:49.210170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.210580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.210980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.239366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:49.241057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:49.345392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b771-0081-4c65-a707-35abab787c93 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.423089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 904/1815] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:31:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:49.424037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.463494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.464193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.464476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.465624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.466615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.469267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.509122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 912/1816] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:49 2022] POST /identity/v3/users => generated 283 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:49.509986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.531882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59db016-e41c-4b8f-b988-3276887ca090 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.536709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.537539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.538036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.538418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.539276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.543827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.563463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6652684-14f0-43a7-b7d5-b7e431902a45 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.577283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.579335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.579866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.581098 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.583506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.587727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.595217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 913/1817] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:49 2022] POST /identity/v3/users => generated 299 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:49.596078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.600904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:49.633144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d93592-68f9-4674-b342-aeaca06ee686 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.643271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.644005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.644663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.644949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.662658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 905/1818] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:49 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:49.662658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.677270 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.679031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.682402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:49.690481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7186bd-8070-4d5e-8349-83e9109af240 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.703913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.704654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.704939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.705213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.705366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 914/1819] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:49.706327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.718012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.718662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd009b-00b7-4a11-a3a5-c0230b323182 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.723569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.724658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.725009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.725293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.726135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.726802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.728187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.742391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 915/1820] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:49.746895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.749438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:49.752780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 906/1821] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:49.756490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.774521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:49.792577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326191dc-28d3-42c1-b55d-da36ebfe052d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.812565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.821832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.822227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.822519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.823327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:49.841296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:49.854891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e272f-1ff5-4d2c-8121-837a94035a01 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.862195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.863073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.864570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.864943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:49.867738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:49.876933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 907/1822] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:49.878486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:49.924923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15d48e-9e83-4a52-89d0-0783d5df2cfe admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:49.942253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:49.943381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:49.943837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:49.944386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] PATH_INFO: `/v3/projects/ef30e1169ce441649bd4bcda60d3380b/users/1812edee4d82437dbe1593c473f18693/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.002072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.003702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef30e1169ce441649bd4bcda60d3380b, user_id=1812edee4d82437dbe1593c473f18693, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.005463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.006646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.027552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 916/1823] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:50.028547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:50.031661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 908/1824] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:49 2022] PUT /identity/v3/projects/ef30e1169ce441649bd4bcda60d3380b/users/1812edee4d82437dbe1593c473f18693/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:50.034071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:50.045932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f117-9814-4c29-a574-29cf00a0e3cf service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:50.144730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:50.145585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.145947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.147017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.148680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.155041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.184642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 917/1825] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:50.184642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:50.189759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8feff-aa27-413a-9810-71e0c351a4a7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:50.203648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:50.204311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.204592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.204872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.205654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.209908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221a95b-33ac-41cb-bd44-54c5a6101487 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:50.211752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.222076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:50.223039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.224587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.225093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.226011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.235277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.257155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 909/1826] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:50.263617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:50.278427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 918/1827] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:50.279384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:50.295880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff342c-1fcd-4c66-8cc7-8bfac945ad87 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:50.309591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce46a2-4003-4575-8188-1d561e47d2b1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:50.311447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:50.313387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.315282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.315837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] PATH_INFO: `/v3/projects/5a350c001c0649b3adf4b1dcbae0a717/users/849564e986124bbc9185ebe01bf5403c/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.332559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ac8b0-c5c9-4df4-bd4a-d021f7b861d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:50.333493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ac8b0-c5c9-4df4-bd4a-d021f7b861d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.333776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ac8b0-c5c9-4df4-bd4a-d021f7b861d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.334048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ac8b0-c5c9-4df4-bd4a-d021f7b861d8 admin admin] PATH_INFO: `/v3/projects/3c4a18dfeb6240d8bbe338bb66f9f727/users/ef361ffb207d4b209ed89c73e49b318a/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.359793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a350c001c0649b3adf4b1dcbae0a717, user_id=849564e986124bbc9185ebe01bf5403c, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.362568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.384696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ac8b0-c5c9-4df4-bd4a-d021f7b861d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4a18dfeb6240d8bbe338bb66f9f727, user_id=ef361ffb207d4b209ed89c73e49b318a, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.399031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 910/1828] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:50 2022] PUT /identity/v3/projects/5a350c001c0649b3adf4b1dcbae0a717/users/849564e986124bbc9185ebe01bf5403c/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:50.399228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ac8b0-c5c9-4df4-bd4a-d021f7b861d8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.399308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:50.431650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 919/1829] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:50 2022] PUT /identity/v3/projects/3c4a18dfeb6240d8bbe338bb66f9f727/users/ef361ffb207d4b209ed89c73e49b318a/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:50.431650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd87f6-259a-4f35-9cef-894304309d62 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.431650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd87f6-259a-4f35-9cef-894304309d62 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.431650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd87f6-259a-4f35-9cef-894304309d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.488111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f8dd87f6-259a-4f35-9cef-894304309d62 None None] MFA Rules not processed for user `849564e986124bbc9185ebe01bf5403c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:50.489284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f8dd87f6-259a-4f35-9cef-894304309d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:50.530910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:50.597197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76fd41-48d5-4ea4-9a29-a7a69ff5d709 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:50.666061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 920/1830] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:50.666061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05dfe-3b58-4020-bc53-f83bccd784fc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.666061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05dfe-3b58-4020-bc53-f83bccd784fc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.666061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea05dfe-3b58-4020-bc53-f83bccd784fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.765912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:50.787939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cea05dfe-3b58-4020-bc53-f83bccd784fc None None] MFA Rules not processed for user `ef361ffb207d4b209ed89c73e49b318a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:50.789663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cea05dfe-3b58-4020-bc53-f83bccd784fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:50.829013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:50.830613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:50.833319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:50.833716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] PATH_INFO: `/v3/projects/ef30e1169ce441649bd4bcda60d3380b/users/1812edee4d82437dbe1593c473f18693/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:50.839239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef30e1169ce441649bd4bcda60d3380b, user_id=1812edee4d82437dbe1593c473f18693, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:50.846264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:50.872344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 911/1831] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:50 2022] PUT /identity/v3/projects/ef30e1169ce441649bd4bcda60d3380b/users/1812edee4d82437dbe1593c473f18693/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:50.873314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.021595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 921/1832] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:51.022797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.023097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.023380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.059768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf8ff9-5ed6-4e7f-8d7a-eee76d3b4baa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.070110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.071938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.071938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.072120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.077500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.079613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.086815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] MFA Rules not processed for user `1812edee4d82437dbe1593c473f18693`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:51.088156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:51.150075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 912/1833] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:50 2022] POST /identity/v3/projects => generated 365 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:51.151345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.204898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 922/1834] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:51.205834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.218757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3620-0995-45dc-b3a6-a8c47a6e95a8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.243772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5768e-9b17-477f-aeca-d7cccbe1d64f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.249957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.250715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.251093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.251444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.252340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.254375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.257251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:51.286452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 913/1835] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:51 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:51.287047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.288752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:51.303367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.304053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.304332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.304616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.305337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.307409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.333597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 923/1836] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:51 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:51.334571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.347035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3842799-b178-45be-b6ef-c251719edce7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.347231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:51.360816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3bee3-2505-4443-85f4-7a8c322c8453 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.364558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.365208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.365535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.365819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.404817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.405244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.405600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.405922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.406660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.409105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.436401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.437884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 924/1837] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:51 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:51.438016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.438925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.447845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 914/1838] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:51.448660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.449777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be809b5e-9415-4f2d-87b2-01832e54964b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.454992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.456337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.456337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.456337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.457028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.458756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.459926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8961c51-16f6-4b0e-99ce-2f60e3149ffe service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.465709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.466320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.466662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.466941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.467873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.469550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.476461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 925/1839] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:51.477726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.485536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 915/1840] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:51.486578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.493278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98464c-87bc-4fe3-9c33-12e4008899a2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.496802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ced2f-d6e2-4a43-8213-77fa162eef04 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.498109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.498974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.499254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.499561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] PATH_INFO: `/v3/projects/cb362da3e854408c9877faaf4154b4e2/users/9efa9fe381f7478a9ef463fe2fe6cf02/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.505283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.506661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.507148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.507792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] PATH_INFO: `/v3/projects/787ce1757ede4a67a2b2734273b96b67/users/dd396f79e77e4d109c585672bafd7eac/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.517928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb362da3e854408c9877faaf4154b4e2, user_id=9efa9fe381f7478a9ef463fe2fe6cf02, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.520711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.529854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787ce1757ede4a67a2b2734273b96b67, user_id=dd396f79e77e4d109c585672bafd7eac, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.532665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.534234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 926/1841] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:51 2022] PUT /identity/v3/projects/cb362da3e854408c9877faaf4154b4e2/users/9efa9fe381f7478a9ef463fe2fe6cf02/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:51.535215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.550933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 916/1842] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:51 2022] PUT /identity/v3/projects/787ce1757ede4a67a2b2734273b96b67/users/dd396f79e77e4d109c585672bafd7eac/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:51.551973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.600028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e68f9-9d3a-4409-9a47-2d5e9bd1f2af admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.614717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ae51c-a030-4dfb-8754-2fb022953403 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.616385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:51.619739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.620413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.620702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.620983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.621816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.624161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.634601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 917/1843] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:51.635355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.645688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179101f5-7876-427e-ae2e-fe766dc4a079 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.650734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.651393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.651677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.651947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.652700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.654387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.660174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.661057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.661337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.661646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] PATH_INFO: `/v3/users/333b149f16d84617b74176918387a270` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.662895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333b149f16d84617b74176918387a270)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.664814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.668573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 918/1844] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:51.668573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.679713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58da22e-d402-444c-8215-f46f1798bad5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.688343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.689354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.689679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.689959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] PATH_INFO: `/v3/projects/cb362da3e854408c9877faaf4154b4e2/users/9efa9fe381f7478a9ef463fe2fe6cf02/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.692101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb362da3e854408c9877faaf4154b4e2, user_id=9efa9fe381f7478a9ef463fe2fe6cf02, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.694876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.710311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 919/1845] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:51 2022] PUT /identity/v3/projects/cb362da3e854408c9877faaf4154b4e2/users/9efa9fe381f7478a9ef463fe2fe6cf02/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:51.719360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:51.731565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333b149f16d84617b74176918387a270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:51.742948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333b149f16d84617b74176918387a270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:51.744276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333b149f16d84617b74176918387a270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:51.745346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333b149f16d84617b74176918387a270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:51.760067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:51.767650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88377b50-ff11-4b63-83b0-561d4ba82ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333b149f16d84617b74176918387a270'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:51.804374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 927/1846] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:51 2022] DELETE /identity/v3/users/333b149f16d84617b74176918387a270 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:51.806201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.808987 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.809877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.815987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b516d-a70d-45d6-b844-19ada0cd6dfe admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:51.854540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] MFA Rules not processed for user `9efa9fe381f7478a9ef463fe2fe6cf02`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:51.856145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:51.904371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896dd41-c0ed-4e68-80cb-ed9a202bab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:51.905844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896dd41-c0ed-4e68-80cb-ed9a202bab84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.906128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896dd41-c0ed-4e68-80cb-ed9a202bab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.906551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896dd41-c0ed-4e68-80cb-ed9a202bab84 admin admin] PATH_INFO: `/v3/projects/787ce1757ede4a67a2b2734273b96b67/users/dd396f79e77e4d109c585672bafd7eac/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.909245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896dd41-c0ed-4e68-80cb-ed9a202bab84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787ce1757ede4a67a2b2734273b96b67, user_id=dd396f79e77e4d109c585672bafd7eac, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:51.913013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896dd41-c0ed-4e68-80cb-ed9a202bab84 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:51.926154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 920/1847] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:51 2022] PUT /identity/v3/projects/787ce1757ede4a67a2b2734273b96b67/users/dd396f79e77e4d109c585672bafd7eac/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:51.927567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39637b40-8366-446d-9aa6-38c88da9b7ef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:51.927859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39637b40-8366-446d-9aa6-38c88da9b7ef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:51.928139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39637b40-8366-446d-9aa6-38c88da9b7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:51.950308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-39637b40-8366-446d-9aa6-38c88da9b7ef None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:51.952167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-39637b40-8366-446d-9aa6-38c88da9b7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.029391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 921/1848] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:52.030935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.031226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.031510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.074358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 928/1849] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:52.086546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.088218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.112577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] MFA Rules not processed for user `dd396f79e77e4d109c585672bafd7eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:52.113838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.158615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:52.197879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a20daa-fcba-41a5-b2e9-969b8ddf0f77 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.206050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:52.222601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 922/1850] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:52.223782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.227993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.276382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a564e26-f59e-45d5-85b8-b506bb16c44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.277198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a564e26-f59e-45d5-85b8-b506bb16c44a service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.277634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a564e26-f59e-45d5-85b8-b506bb16c44a service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.278033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a564e26-f59e-45d5-85b8-b506bb16c44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.291507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a564e26-f59e-45d5-85b8-b506bb16c44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.293461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a564e26-f59e-45d5-85b8-b506bb16c44a service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.311054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 929/1851] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:52.312974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76338-cc5d-4959-bbda-f77bbf26dadd None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.313448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76338-cc5d-4959-bbda-f77bbf26dadd None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.314009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76338-cc5d-4959-bbda-f77bbf26dadd None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.315646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 930/1852] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:31:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:31:52.329387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.329882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.330345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.346775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164d32-ec2d-4386-b336-bd899046b9ad None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.352401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.354367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.354732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.355084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.365624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:52.369655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.471228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.481860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.486149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 931/1853] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:31:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:52.487497 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.488841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.503394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e1da8-8940-47bc-ac6d-b81ee1e967a6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.518714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 923/1854] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:52.523387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.526079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.526745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.526745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.552259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.553766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.576410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 932/1855] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:52.608578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.610607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.630206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bd698-8cfe-4f23-9e69-01da39940a99 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.641498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.642168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.642452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.642730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.656721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.658461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.682775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 924/1856] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:52.709188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.721379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe9e46-c7a6-4cd2-bdc5-b841b31f4fc6 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.729403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.730094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.730419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.730699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.733046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.804005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.805726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.812064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:52.824227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.827636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 933/1857] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:52.835235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece83d7-e57f-442d-bb08-c234fd04469d service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.839778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.840567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.840985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.841386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.843368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.898579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:52.900132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:52.917376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6dd42-cc06-4093-ba87-9ac5f194c1f5 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:52.923888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.924446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:52.925521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.925655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:52.926177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:52.926578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] PATH_INFO: `/v3/projects/dfcc6ae6e0d44913a0766690d7aea0cb` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:52.928286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfcc6ae6e0d44913a0766690d7aea0cb)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:52.932612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:52.959693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 925/1858] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:53.086614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfcc6ae6e0d44913a0766690d7aea0cb', 'request_id': 'req-10e240c9-894d-4c07-9c65-0afc0084aed7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.109681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfcc6ae6e0d44913a0766690d7aea0cb', 'request_id': 'req-10e240c9-894d-4c07-9c65-0afc0084aed7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.130618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfcc6ae6e0d44913a0766690d7aea0cb', 'request_id': 'req-10e240c9-894d-4c07-9c65-0afc0084aed7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.161223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] The token cache is being invalidate because project dfcc6ae6e0d44913a0766690d7aea0cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:53.161618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.162602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.166119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 934/1859] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:52 2022] DELETE /identity/v3/projects/dfcc6ae6e0d44913a0766690d7aea0cb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:53.170454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:53.188529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4927548-95c3-44fa-aec9-6381393f8089 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:53.254256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:53.299033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:53.300332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:53.305039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:53.305039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] PATH_INFO: `/v3/users/1812edee4d82437dbe1593c473f18693` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:53.305835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1812edee4d82437dbe1593c473f18693)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:53.309757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:53.405875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1812edee4d82437dbe1593c473f18693'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.437583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1812edee4d82437dbe1593c473f18693'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.438528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1812edee4d82437dbe1593c473f18693'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.439161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1812edee4d82437dbe1593c473f18693'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.463256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f385f71b-a606-4073-825b-19c2be9e187b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1812edee4d82437dbe1593c473f18693'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.491036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 926/1860] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:53 2022] DELETE /identity/v3/users/1812edee4d82437dbe1593c473f18693 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:53.641345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:53.647357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:53.682180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e240c9-894d-4c07-9c65-0afc0084aed7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:53.810006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:53.811150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:53.811453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:53.811832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] PATH_INFO: `/v3/users/ef361ffb207d4b209ed89c73e49b318a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:53.813266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef361ffb207d4b209ed89c73e49b318a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:53.817704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:53.910880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef361ffb207d4b209ed89c73e49b318a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.937198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef361ffb207d4b209ed89c73e49b318a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.938205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef361ffb207d4b209ed89c73e49b318a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.939795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef361ffb207d4b209ed89c73e49b318a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.956700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef361ffb207d4b209ed89c73e49b318a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:53.977966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 935/1861] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:53 2022] DELETE /identity/v3/users/ef361ffb207d4b209ed89c73e49b318a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:53.982420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:53.982919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:53.983175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.017984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:54.020098 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.132102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 927/1862] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:54.142936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.144125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.150297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:54.236031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a19e1d-7c7e-4aae-8421-07e7396551b8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.250085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41089441-337a-41fa-bdb3-fe86dae1873b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.250799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41089441-337a-41fa-bdb3-fe86dae1873b service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.251089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41089441-337a-41fa-bdb3-fe86dae1873b service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.251361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41089441-337a-41fa-bdb3-fe86dae1873b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.272206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41089441-337a-41fa-bdb3-fe86dae1873b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.273833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41089441-337a-41fa-bdb3-fe86dae1873b service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.294352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 936/1863] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:54.308167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.310317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.331902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf89749-0f1b-4802-a84e-dc71e9b23837 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.338189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.339437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.340058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.340586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] PATH_INFO: `/v3/projects/ef30e1169ce441649bd4bcda60d3380b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.342315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef30e1169ce441649bd4bcda60d3380b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.344864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.437439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:54.440617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef30e1169ce441649bd4bcda60d3380b', 'request_id': 'req-841b23eb-94c2-4c1a-9833-e8d54e9993cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.450648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef30e1169ce441649bd4bcda60d3380b', 'request_id': 'req-841b23eb-94c2-4c1a-9833-e8d54e9993cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.458393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef30e1169ce441649bd4bcda60d3380b', 'request_id': 'req-841b23eb-94c2-4c1a-9833-e8d54e9993cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.473256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] The token cache is being invalidate because project ef30e1169ce441649bd4bcda60d3380b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:54.473618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.474402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-841b23eb-94c2-4c1a-9833-e8d54e9993cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.477534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 928/1864] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:54 2022] DELETE /identity/v3/projects/ef30e1169ce441649bd4bcda60d3380b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:54.484323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.485617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.486252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.518975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:54.520246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.628113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 937/1865] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:54.636023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.636518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.636992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.660410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:54.661762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.671919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 929/1866] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:54.677402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.691910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9de574-a5c7-4a3e-b10f-c58bd55ea030 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.708235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.709685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.719235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.719996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.720355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.720796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.721656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:54.722078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.725064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.728500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afb11f-95e3-4ed4-9e2b-57fdb7fe9c94 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.743802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.744475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.744763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.745068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] PATH_INFO: `/v3/projects/3c4a18dfeb6240d8bbe338bb66f9f727` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.746575 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4a18dfeb6240d8bbe338bb66f9f727)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.749071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.752364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 938/1867] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:54.762564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.780886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af98cc-86d1-4420-b333-4030886d1393 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.791966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.792645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.792938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.793939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.793939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.796099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.858913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4a18dfeb6240d8bbe338bb66f9f727', 'request_id': 'req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.871385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 939/1868] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:54 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:54.876070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4a18dfeb6240d8bbe338bb66f9f727', 'request_id': 'req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.877796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.888239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4a18dfeb6240d8bbe338bb66f9f727', 'request_id': 'req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.902315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] The token cache is being invalidate because project 3c4a18dfeb6240d8bbe338bb66f9f727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:54.902824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.903586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:54.905666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 930/1869] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:54 2022] DELETE /identity/v3/projects/3c4a18dfeb6240d8bbe338bb66f9f727 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:54.909180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.910611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:54.951035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c02372-0426-49d0-888c-bda3df8c5ca3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.955951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.956616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.956899 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.957176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.957894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.959811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.977509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:54.980291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15f5f1-7bd8-4ec6-ae0e-752d271fdb22 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:54.984259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 940/1870] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:54 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:31:54.985871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:54.986765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:54.987058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:54.987334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] PATH_INFO: `/v3/users/dd396f79e77e4d109c585672bafd7eac` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:54.988533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:54.988958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd396f79e77e4d109c585672bafd7eac)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:54.991121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:54.992244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:55.006724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd45e6f-04b3-4c38-9908-f03d02ade3bf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.012231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.012901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.015068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.015068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.015068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.016137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.032225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 941/1871] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:55.036305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.059842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51294fff-d7bb-443b-9c9d-9f840e5d2df9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.061358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd396f79e77e4d109c585672bafd7eac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:55.066305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.066612 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:55.068010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.068437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.068869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.070292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.072127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.076295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd396f79e77e4d109c585672bafd7eac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:55.077458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd396f79e77e4d109c585672bafd7eac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:55.078477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd396f79e77e4d109c585672bafd7eac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:55.088984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ff885c9-a1e8-4e01-855c-efcb56a4fe71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd396f79e77e4d109c585672bafd7eac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:55.090351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 942/1872] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:55.091357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.094598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:55.106176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 931/1873] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:54 2022] DELETE /identity/v3/users/dd396f79e77e4d109c585672bafd7eac => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:55.110595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac1e0a-9a97-4fd4-b052-4496cc900c8e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.206296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.208178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.208704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.209164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] PATH_INFO: `/v3/projects/cf9196b849294bdc8f73c8a1cdd7e2cb/users/f53be47d2ff742a7874ecd6d85566d2c/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.233400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9196b849294bdc8f73c8a1cdd7e2cb, user_id=f53be47d2ff742a7874ecd6d85566d2c, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.238796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.258893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 943/1874] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:55 2022] PUT /identity/v3/projects/cf9196b849294bdc8f73c8a1cdd7e2cb/users/f53be47d2ff742a7874ecd6d85566d2c/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:55.279819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.280343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.280984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.328880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] MFA Rules not processed for user `f53be47d2ff742a7874ecd6d85566d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:55.330652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:55.439517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 932/1875] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:55.441099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.549622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3381e-25b0-40ef-ab55-195153785e20 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.558285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.559294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.559588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.559869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.560616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.569735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.626928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.627678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 944/1876] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:55 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:55.640346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42229f2e-dbd3-4051-92ed-cadae5982878 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.722442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.723136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.723424 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.723702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.724412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.726445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.767189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 933/1877] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:55 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:31:55.767531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.779560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba740610-ec2a-4b38-84e1-06a14f3255c9 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.785646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.786571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.786998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.788090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.789022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.791685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.809697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 945/1878] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:31:55.811480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.837598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4410b-8d1a-4295-b84e-ea0b830fd457 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:55.848666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:55.849687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:55.849988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:55.850251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] PATH_INFO: `/v3/projects/68f77acdaca34dab9cc7de3cfa29af3f/users/187fa392d61e45a9bcc24a2a3d4ae4fc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:55.876425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f77acdaca34dab9cc7de3cfa29af3f, user_id=187fa392d61e45a9bcc24a2a3d4ae4fc, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:55.880724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:55.905080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 934/1879] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:55 2022] PUT /identity/v3/projects/68f77acdaca34dab9cc7de3cfa29af3f/users/187fa392d61e45a9bcc24a2a3d4ae4fc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:55.909131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:55.928115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2a61-4fb4-48c6-946a-7d7671d6dffc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:56.054784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:56.116336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:56.117313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.117841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.118273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.119189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:56.123022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:56.139716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 946/1880] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:56.142949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:56.144840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.149699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:56.166645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f65e07-a1c8-449a-828f-ce0b521a6ab0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:56.175456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89684861-685f-4376-998a-377d31f0afa4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:56.179204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:56.181403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.182784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.183228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] PATH_INFO: `/v3/users/21611090b48245d9930a710f1aea630c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.184957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21611090b48245d9930a710f1aea630c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:56.187659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:56.192398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:56.196549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.196815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.197148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] PATH_INFO: `/v3/projects/68f77acdaca34dab9cc7de3cfa29af3f/users/187fa392d61e45a9bcc24a2a3d4ae4fc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.201639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f77acdaca34dab9cc7de3cfa29af3f, user_id=187fa392d61e45a9bcc24a2a3d4ae4fc, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:56.209132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:56.226416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:56.236161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 947/1881] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:56 2022] PUT /identity/v3/projects/68f77acdaca34dab9cc7de3cfa29af3f/users/187fa392d61e45a9bcc24a2a3d4ae4fc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:56.237707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:56.239813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.322663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21611090b48245d9930a710f1aea630c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.344518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21611090b48245d9930a710f1aea630c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.345264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21611090b48245d9930a710f1aea630c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.346368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21611090b48245d9930a710f1aea630c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.355641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-490c862e-30e5-4dc4-b844-4f9c77ae1df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21611090b48245d9930a710f1aea630c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.367948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6e766-f37d-4b62-a70e-9d335395e340 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:56.373749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 935/1882] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:56 2022] DELETE /identity/v3/users/21611090b48245d9930a710f1aea630c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:56.374501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:56.375283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6c251-881b-4d03-8d6b-85c9e780a04a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.375580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6c251-881b-4d03-8d6b-85c9e780a04a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.375902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6c251-881b-4d03-8d6b-85c9e780a04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.376292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.376713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.377128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] PATH_INFO: `/v3/projects/787ce1757ede4a67a2b2734273b96b67` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.379551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787ce1757ede4a67a2b2734273b96b67)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:56.385330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:56.415771 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-77f6c251-881b-4d03-8d6b-85c9e780a04a None None] MFA Rules not processed for user `187fa392d61e45a9bcc24a2a3d4ae4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:56.418204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-77f6c251-881b-4d03-8d6b-85c9e780a04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.485993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787ce1757ede4a67a2b2734273b96b67', 'request_id': 'req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.498867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787ce1757ede4a67a2b2734273b96b67', 'request_id': 'req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.508286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787ce1757ede4a67a2b2734273b96b67', 'request_id': 'req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.541822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] The token cache is being invalidate because project 787ce1757ede4a67a2b2734273b96b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:56.542052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.546664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4138d48-0b3c-4ce4-a29b-8f1877fd16f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:56.552781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 948/1883] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:56 2022] DELETE /identity/v3/projects/787ce1757ede4a67a2b2734273b96b67 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:56.553068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 936/1884] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:56.557311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.557632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.557916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.558615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.559015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.560358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.609352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:56.610432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:56.611780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.618840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.689992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:56.730648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 949/1885] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:56.738479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:56.741088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.779317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 937/1886] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:56.779317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:56.781575 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.836079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081bf9f-f9f6-411b-86d3-4ffbe5f10757 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:56.843192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:56.844278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:56.846955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.847448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.847931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.853238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.899974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cc61-cd48-437b-85a8-0233f1e85a20 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:56.903984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08a5ec-980f-4abb-929d-9f7405e27d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:56.904838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08a5ec-980f-4abb-929d-9f7405e27d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.905123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08a5ec-980f-4abb-929d-9f7405e27d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.905403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08a5ec-980f-4abb-929d-9f7405e27d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.920061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08a5ec-980f-4abb-929d-9f7405e27d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:56.922376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08a5ec-980f-4abb-929d-9f7405e27d5f service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:56.939317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 938/1887] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:56.941403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:56.941958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:56.943261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:56.944241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:56.945960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:56.971342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 950/1888] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:56.976039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:56.977260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:56.990047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 939/1889] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:56.994435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.012162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b6cc-7044-4221-86cf-c3e398a124fc service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.018545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.019504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.019791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.020124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.022919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:57.023243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.025240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.043003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 951/1890] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:57.046826 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.064993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ee4fc-832b-4e7d-ad78-2714302d4f96 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.070738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.071396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.071753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.072223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.072945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.074975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.122292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 940/1891] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:57 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:57.126356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.143834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c0fa-82c8-40e9-a015-fa3bbbf541bf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.290172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.290915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.291205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.291795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.292504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.296693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.359938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 952/1892] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:57 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:57.364304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.378135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6633272-7a2f-4a94-af4d-58820b35b797 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.388090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.390153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.391059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.391588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.394719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.397775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.417386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 941/1893] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:57.421605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.433077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731595d-55b8-41c3-bd87-df92655db347 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.439834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.440514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.440796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.441353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.442089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.444515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.457483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 953/1894] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:57.463530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.476869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e6a3-53c4-4ef5-ae53-8b04a351aded admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.482718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac17fe-2688-43cb-8bc9-0c77b165c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.483655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac17fe-2688-43cb-8bc9-0c77b165c0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.484886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac17fe-2688-43cb-8bc9-0c77b165c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.485233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac17fe-2688-43cb-8bc9-0c77b165c0dd admin admin] PATH_INFO: `/v3/projects/2e148a0e86cb4229a06bf3afcb314db1/users/98df8d75ee16460e9ff2bb415c8782c8/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.502296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.504283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:57.518699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:57.520014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac17fe-2688-43cb-8bc9-0c77b165c0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e148a0e86cb4229a06bf3afcb314db1, user_id=98df8d75ee16460e9ff2bb415c8782c8, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.522892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac17fe-2688-43cb-8bc9-0c77b165c0dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.523181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e839-f9fd-42f7-a424-227b52dbb5ea admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.537477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.540066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.540481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.541920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] PATH_INFO: `/v3/projects/8493133d405443efb4451fe7415ab675` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.543786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8493133d405443efb4451fe7415ab675)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.548269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.553260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 942/1895] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:57 2022] PUT /identity/v3/projects/2e148a0e86cb4229a06bf3afcb314db1/users/98df8d75ee16460e9ff2bb415c8782c8/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:57.565445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.566242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.566242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.605760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] MFA Rules not processed for user `98df8d75ee16460e9ff2bb415c8782c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:57.607548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:57.611235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8493133d405443efb4451fe7415ab675', 'request_id': 'req-7704291b-420b-41d7-b08f-50cf01602bc0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.623990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8493133d405443efb4451fe7415ab675', 'request_id': 'req-7704291b-420b-41d7-b08f-50cf01602bc0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.638750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8493133d405443efb4451fe7415ab675', 'request_id': 'req-7704291b-420b-41d7-b08f-50cf01602bc0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.654039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] The token cache is being invalidate because project 8493133d405443efb4451fe7415ab675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:57.654625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.655484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.657925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 954/1896] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:57 2022] DELETE /identity/v3/projects/8493133d405443efb4451fe7415ab675 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:57.659432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.661667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:57.694659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 943/1897] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:57.695637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.696979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:57.740172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704291b-420b-41d7-b08f-50cf01602bc0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.745341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '187fa392d61e45a9bcc24a2a3d4ae4fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f77acdaca34dab9cc7de3cfa29af3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.745946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.746360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.746640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.747376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:57.747750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.749346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.765720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 955/1898] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:57.766296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.771948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:57.775399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:57.804572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa1864-3357-4c20-8409-be4181933e7e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.818559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.819440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.819726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.820067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] PATH_INFO: `/v3/users/8354bbff30164fc3a9778f9bb2828ca0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.822854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8354bbff30164fc3a9778f9bb2828ca0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.824809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.897721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13f1fc-8d76-4f4f-b91a-d12f517dadc0 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:57.903051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8354bbff30164fc3a9778f9bb2828ca0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.903747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:57.904589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:57.904867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:57.905142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:57.906143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:57.908902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:57.925384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8354bbff30164fc3a9778f9bb2828ca0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.926474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8354bbff30164fc3a9778f9bb2828ca0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.927226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8354bbff30164fc3a9778f9bb2828ca0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.947853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 956/1899] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:57 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:57.950403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.953137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:57.954066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8354bbff30164fc3a9778f9bb2828ca0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:57.965812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:57.985270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 944/1900] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:57 2022] DELETE /identity/v3/users/8354bbff30164fc3a9778f9bb2828ca0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:57.986434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:57.987836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.047073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d500f-6d12-4700-9abf-948df4c946c2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.070326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c778b6-9fc8-4293-8dc7-50fc445acab2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.075458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.076373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.076661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.077325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.078069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.080027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.102190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 945/1901] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:57 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:58.107310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.119764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ca1a1-52e6-43bd-bd25-2577b30522fa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.120742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '187fa392d61e45a9bcc24a2a3d4ae4fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f77acdaca34dab9cc7de3cfa29af3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.121491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.121845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.122217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.123137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.125310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.125515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.126277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.126568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.126923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.127659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.129468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.141507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 946/1902] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:58.145845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.153658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 957/1903] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:57 2022] POST /identity/v3/projects => generated 353 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:58.157391 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.159407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca279-5a44-44e7-895c-a053221bc431 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.215934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:58.230148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b60eaf-648d-472b-ae64-d649a64a4efb tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.234489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.236980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '187fa392d61e45a9bcc24a2a3d4ae4fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f77acdaca34dab9cc7de3cfa29af3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.237584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.237948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.238242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.238659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.239100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.239403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:58.239710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.241294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] PATH_INFO: `/v3/projects/acf3d086c6d04304b7c487f18e77dd46/users/511d12b1ba5c441e95bda62365a671ec/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.241846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.258075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 958/1904] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:31:58.264501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf3d086c6d04304b7c487f18e77dd46, user_id=511d12b1ba5c441e95bda62365a671ec, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.267954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.286119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 947/1905] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:58 2022] PUT /identity/v3/projects/acf3d086c6d04304b7c487f18e77dd46/users/511d12b1ba5c441e95bda62365a671ec/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:58.290292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.305301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb8dd-b465-4385-a055-c6d9948f2e64 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.306506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.308310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.345454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:58.374295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.374973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.375258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.375537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.376275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.377929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.391306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 959/1906] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:58.391306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.391550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c154e0-984e-430a-80b2-796e3178b5a7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.394520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.395390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.395673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.395962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] PATH_INFO: `/v3/projects/4bd72c953cfc4fd9844500f1d5f7795d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.397136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd72c953cfc4fd9844500f1d5f7795d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.399091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.445770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd72c953cfc4fd9844500f1d5f7795d', 'request_id': 'req-f73b0c48-6474-46a5-aacc-ebc303713b83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.452065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512db9-c291-450c-9843-279b7e0f25d7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.455124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd72c953cfc4fd9844500f1d5f7795d', 'request_id': 'req-f73b0c48-6474-46a5-aacc-ebc303713b83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.462601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd72c953cfc4fd9844500f1d5f7795d', 'request_id': 'req-f73b0c48-6474-46a5-aacc-ebc303713b83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.473762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] The token cache is being invalidate because project 4bd72c953cfc4fd9844500f1d5f7795d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:58.474050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.474790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.476835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 948/1907] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:58 2022] DELETE /identity/v3/projects/4bd72c953cfc4fd9844500f1d5f7795d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:58.477315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.478836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.512012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '187fa392d61e45a9bcc24a2a3d4ae4fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f77acdaca34dab9cc7de3cfa29af3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.512891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.513179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.513570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] PATH_INFO: `/v3/projects/09eee2d5cf42441da713a3ae1949a949` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.514826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorizing `identity:delete_project(project_id=09eee2d5cf42441da713a3ae1949a949)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.516752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.544845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0c48-6474-46a5-aacc-ebc303713b83 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.550640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.551533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.551882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.552157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] PATH_INFO: `/v3/projects/acf3d086c6d04304b7c487f18e77dd46/users/511d12b1ba5c441e95bda62365a671ec/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.554191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf3d086c6d04304b7c487f18e77dd46, user_id=511d12b1ba5c441e95bda62365a671ec, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.556879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.560078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:58.565688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09eee2d5cf42441da713a3ae1949a949', 'request_id': 'req-bd05c128-6a5f-46b2-9a2a-48923ece9165'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.570961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 949/1908] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:58 2022] PUT /identity/v3/projects/acf3d086c6d04304b7c487f18e77dd46/users/511d12b1ba5c441e95bda62365a671ec/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:58.572288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.574324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.582925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09eee2d5cf42441da713a3ae1949a949', 'request_id': 'req-bd05c128-6a5f-46b2-9a2a-48923ece9165'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.589459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09eee2d5cf42441da713a3ae1949a949', 'request_id': 'req-bd05c128-6a5f-46b2-9a2a-48923ece9165'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.601161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] The token cache is being invalidate because project 09eee2d5cf42441da713a3ae1949a949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:58.601484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.603805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-bd05c128-6a5f-46b2-9a2a-48923ece9165 tempest-VolumeQuotaClassesTest-1450977478 tempest-VolumeQuotaClassesTest-1450977478-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.605297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 960/1909] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:58 2022] DELETE /identity/v3/projects/09eee2d5cf42441da713a3ae1949a949 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:58.606589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.606876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.607231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.618994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:58.641102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] MFA Rules not processed for user `511d12b1ba5c441e95bda62365a671ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:58.643003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.677024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7426f-241a-421e-ab73-d3fdea693bce admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.682095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.682976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.683948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.683948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] PATH_INFO: `/v3/users/a20d3c1ee68b434788bd1d8e4c1e4d85` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.684891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20d3c1ee68b434788bd1d8e4c1e4d85)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.686947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.724710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 961/1910] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:58.738496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.740060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.741378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20d3c1ee68b434788bd1d8e4c1e4d85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.756965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20d3c1ee68b434788bd1d8e4c1e4d85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.757735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20d3c1ee68b434788bd1d8e4c1e4d85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.758323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20d3c1ee68b434788bd1d8e4c1e4d85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.766728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20d3c1ee68b434788bd1d8e4c1e4d85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:58.782494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 950/1911] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:58 2022] DELETE /identity/v3/users/a20d3c1ee68b434788bd1d8e4c1e4d85 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:58.810371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0952822-c150-43a8-9fa8-efbdac9cc5de None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:58.868469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:58.869196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:58.869463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:58.869747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:58.871792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.911234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:58.912729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:58.945439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:58.947600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:58.954229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:58.959928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 962/1912] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:31:59.006391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67913bf-6a4a-47fe-a43f-e04dbecfa655 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.010911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.011797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.012073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.012345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] PATH_INFO: `/v3/users/f53be47d2ff742a7874ecd6d85566d2c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.013849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53be47d2ff742a7874ecd6d85566d2c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.016961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.037963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:59.064810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.067311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.089176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9b91d-88ee-4ddc-951d-2df2802a3c33 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.096183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.096852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.097136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.097440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.100487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53be47d2ff742a7874ecd6d85566d2c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.105036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.124422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53be47d2ff742a7874ecd6d85566d2c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.125561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53be47d2ff742a7874ecd6d85566d2c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.126375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53be47d2ff742a7874ecd6d85566d2c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.135761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53be47d2ff742a7874ecd6d85566d2c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.156200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 951/1913] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:31:58 2022] DELETE /identity/v3/users/f53be47d2ff742a7874ecd6d85566d2c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:31:59.157308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.160196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.215184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.217950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ded78-6b1f-46e5-8180-07053453e261 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.228031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 963/1914] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:59.229644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.230028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.230215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.250739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:59.252024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.270653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba086cf-1c95-4b1f-9275-9d16401247c1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.276711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.278016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.278273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.278559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] PATH_INFO: `/v3/projects/2ce701bafcc64ed4915adf8b7337b4b0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.280213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce701bafcc64ed4915adf8b7337b4b0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.282485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.336867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 964/1915] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:59.348829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce701bafcc64ed4915adf8b7337b4b0', 'request_id': 'req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.351367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.353944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.359103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce701bafcc64ed4915adf8b7337b4b0', 'request_id': 'req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.374475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce701bafcc64ed4915adf8b7337b4b0', 'request_id': 'req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.386803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] The token cache is being invalidate because project 2ce701bafcc64ed4915adf8b7337b4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:31:59.387159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.387887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3dee9-22a0-4dad-ba6d-cdbc763c307e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:31:59.390061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 952/1916] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:59 2022] DELETE /identity/v3/projects/2ce701bafcc64ed4915adf8b7337b4b0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:59.397067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167c52-8ee1-4aaf-9781-b89875447899 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.397727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167c52-8ee1-4aaf-9781-b89875447899 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.397727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5167c52-8ee1-4aaf-9781-b89875447899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.431816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d5167c52-8ee1-4aaf-9781-b89875447899 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:59.433130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d5167c52-8ee1-4aaf-9781-b89875447899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.455454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faccab-9732-4ee1-b461-45ae3ab88be1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.456607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:59.511278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.519446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.519598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 953/1917] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:31:59.519694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.519694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.521167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.521296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.521374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.521743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.534022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:31:59.537054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.538664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.543603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:59.545097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.554873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 954/1918] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:31:59.558321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.561206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 965/1919] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:31:59.570804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047044b9-f7ee-4472-8e5d-e6cb83637eab None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.575330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.576152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.576434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.576700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.577731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:31:59.578036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.579655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.595060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 955/1920] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:31:59.599733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.610690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02914313-6181-4359-a387-da17f6e074bd service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.615196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.615911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.616126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.616397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.617117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.619063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.652052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 966/1921] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:31:59 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:59.657352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.676936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4d672-45b4-49ce-ab76-eaa5a9b016a4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.754252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.755059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.755383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.755746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.756519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.758590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.780442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 956/1922] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:31:59 2022] POST /identity/v3/users => generated 305 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:31:59.784591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.796155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0ce86-13b3-42d1-800e-4d9b2b42adeb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.801800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.802364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.802988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.803168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.803951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.805721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.817445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 967/1923] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:59.821666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.832539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915038cc-2242-48e5-a883-59a6bedf8845 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.837837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.838507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.838805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.839080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.839803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.841558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.855185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 957/1924] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:31:59.856645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.867782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076d6ea-a350-4ff0-8156-69eb88b64fbf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.873848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eacc2-aedd-43dc-82a9-9adf7917ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.874742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eacc2-aedd-43dc-82a9-9adf7917ba34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.875049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eacc2-aedd-43dc-82a9-9adf7917ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.875321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eacc2-aedd-43dc-82a9-9adf7917ba34 admin admin] PATH_INFO: `/v3/projects/01377371ddd74cb990b95aec171e2f14/users/98898eedb0d843c18aa24fb7fd60d52e/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.882121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:31:59.884503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.893253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eacc2-aedd-43dc-82a9-9adf7917ba34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01377371ddd74cb990b95aec171e2f14, user_id=98898eedb0d843c18aa24fb7fd60d52e, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.896020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eacc2-aedd-43dc-82a9-9adf7917ba34 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:31:59.913953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbde4e-b5b0-4a1a-b59f-de02dd378abf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:31:59.924408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 968/1925] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:31:59 2022] PUT /identity/v3/projects/01377371ddd74cb990b95aec171e2f14/users/98898eedb0d843c18aa24fb7fd60d52e/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:31:59.938506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.938990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.939272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.967453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] MFA Rules not processed for user `98898eedb0d843c18aa24fb7fd60d52e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:31:59.968690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:31:59.975962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:31:59.976838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:31:59.977124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:31:59.977398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] PATH_INFO: `/v3/projects/cf9196b849294bdc8f73c8a1cdd7e2cb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:31:59.979088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf9196b849294bdc8f73c8a1cdd7e2cb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:31:59.981183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.012607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:00.030297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf9196b849294bdc8f73c8a1cdd7e2cb', 'request_id': 'req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.035700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 969/1926] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:00.038132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf9196b849294bdc8f73c8a1cdd7e2cb', 'request_id': 'req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.040022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.045530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf9196b849294bdc8f73c8a1cdd7e2cb', 'request_id': 'req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.051901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135642a-25d2-4cc8-9478-0ab446df8c45 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.056424 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] The token cache is being invalidate because project cf9196b849294bdc8f73c8a1cdd7e2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:00.056712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.057364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.058940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 958/1927] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:31:59 2022] DELETE /identity/v3/projects/cf9196b849294bdc8f73c8a1cdd7e2cb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:00.062730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.064030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.111259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.111844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.112145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.112421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.113126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.115085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.129084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9cca7-aa22-48ef-a0ad-ce1134fea16a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.135425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.136311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.136599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.136894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] PATH_INFO: `/v3/users/187fa392d61e45a9bcc24a2a3d4ae4fc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.138186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] RBAC: Authorizing `identity:delete_user(user_id=187fa392d61e45a9bcc24a2a3d4ae4fc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.140963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.144169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 970/1928] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:00 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:00.145210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.147052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.161089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:00.191540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187fa392d61e45a9bcc24a2a3d4ae4fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.205974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187fa392d61e45a9bcc24a2a3d4ae4fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.206789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187fa392d61e45a9bcc24a2a3d4ae4fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.207493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187fa392d61e45a9bcc24a2a3d4ae4fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.215850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187fa392d61e45a9bcc24a2a3d4ae4fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.217218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11316035-47b8-4a95-b05a-282c46d0a9b2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.222652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.223347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.223660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.223935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.224638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.228352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.237820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 959/1929] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:00 2022] DELETE /identity/v3/users/187fa392d61e45a9bcc24a2a3d4ae4fc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:00.256135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 971/1930] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:00 2022] POST /identity/v3/users => generated 311 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:00.258568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.260086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.311867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:00.356395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b275a-8d16-4d91-a727-e14f002c703a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.362010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.362710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.363064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.363419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.364456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.366201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.381332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 960/1931] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:00.384375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.395779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4c2f3-b647-449b-b486-b8c71a4d3650 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.405962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.406871 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.407158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.407435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] PATH_INFO: `/v3/projects/5ec84ae9d38143f98e69ba431ace94fa/users/bceb6c43b2774e0aac9344d42de01efe/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.427526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec84ae9d38143f98e69ba431ace94fa, user_id=bceb6c43b2774e0aac9344d42de01efe, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.430757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.457390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 972/1932] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:00 2022] PUT /identity/v3/projects/5ec84ae9d38143f98e69ba431ace94fa/users/bceb6c43b2774e0aac9344d42de01efe/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:00.461462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.478440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0610d72-d39d-40cf-a671-67ed7800faaa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.551436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a737ae8-f30e-4822-a43a-72e3395a1c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.552124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737ae8-f30e-4822-a43a-72e3395a1c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.552406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737ae8-f30e-4822-a43a-72e3395a1c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.552678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a737ae8-f30e-4822-a43a-72e3395a1c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.553845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737ae8-f30e-4822-a43a-72e3395a1c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.555498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a737ae8-f30e-4822-a43a-72e3395a1c23 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.566580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 961/1933] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:00.571010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.582161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d85c8-9195-4510-a378-8809888189bd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.586906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.587804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.588087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.588413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] PATH_INFO: `/v3/projects/5ec84ae9d38143f98e69ba431ace94fa/users/bceb6c43b2774e0aac9344d42de01efe/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.590500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec84ae9d38143f98e69ba431ace94fa, user_id=bceb6c43b2774e0aac9344d42de01efe, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.593723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.606184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 973/1934] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:00 2022] PUT /identity/v3/projects/5ec84ae9d38143f98e69ba431ace94fa/users/bceb6c43b2774e0aac9344d42de01efe/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:00.618883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.619132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.619442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.629360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.634590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.666827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] MFA Rules not processed for user `bceb6c43b2774e0aac9344d42de01efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:00.668534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.721171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e05bb2-47df-4cf4-861a-f666d698f64b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.726769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.727719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.728203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.728568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] PATH_INFO: `/v3/projects/68f77acdaca34dab9cc7de3cfa29af3f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.729870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f77acdaca34dab9cc7de3cfa29af3f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:00.731935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:00.754440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 962/1935] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:00 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:00.763664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:00.765214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.791818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f77acdaca34dab9cc7de3cfa29af3f', 'request_id': 'req-a1154709-6e0a-4c1c-a01b-f84db6bd123e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.801738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f77acdaca34dab9cc7de3cfa29af3f', 'request_id': 'req-a1154709-6e0a-4c1c-a01b-f84db6bd123e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.817669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f77acdaca34dab9cc7de3cfa29af3f', 'request_id': 'req-a1154709-6e0a-4c1c-a01b-f84db6bd123e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.820686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:00.835219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] The token cache is being invalidate because project 68f77acdaca34dab9cc7de3cfa29af3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:00.836195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.839907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a1154709-6e0a-4c1c-a01b-f84db6bd123e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:00.843056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 974/1936] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:00 2022] DELETE /identity/v3/projects/68f77acdaca34dab9cc7de3cfa29af3f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:00.847576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8963-a2c5-4f98-a769-8d675995dcf2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.847999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8963-a2c5-4f98-a769-8d675995dcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.848427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8963-a2c5-4f98-a769-8d675995dcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.858526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497741-f9b0-4396-82dd-43088b7b5bce None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:00.864385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:00.880650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b8df8963-a2c5-4f98-a769-8d675995dcf2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:00.881738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b8df8963-a2c5-4f98-a769-8d675995dcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.941306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:00.942030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.942317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.942863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:00.946835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:00.963239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 975/1937] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:00.970822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:00.971435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:00.971934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.007422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:01.008827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:01.023070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 976/1938] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:01.025620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.052601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f50cee-7b6a-4c8e-bfd5-3e598984730b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.062265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.063113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.063452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.063748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.065641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:01.066091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.072221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.077048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.080554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.101123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 963/1939] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:01.102515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.110121 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 977/1940] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:32:01.116023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db559feb-604a-44d3-b34f-2bd528411cc8 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.122162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.123151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.123539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.123986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.124791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.127255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.184631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 964/1941] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:01 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:01.188806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.212334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26a2c1-1db9-4ea2-b141-6731624f04af admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.336993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.338577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.339603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.340081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.343287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.348124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.385977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 978/1942] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:01 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:01.389868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.402594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ac27d-7541-4e61-9730-91f290333e59 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.415034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.417562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.418702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.419054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.432784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.441023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.441210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.442915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:01.463820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 965/1943] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:01.469463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.491999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a7af6-3673-411b-9e5f-791aab5736bd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.502590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.503283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.503563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.503838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.504573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.506252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.539592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 966/1944] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:01.540869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.568714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26a231-6c81-4601-8646-317f96722e79 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.583151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf336-765f-4d49-800b-3d9c0d0a70f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.583904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822e874-091f-42eb-8182-b4858460a469 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.584067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf336-765f-4d49-800b-3d9c0d0a70f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.584324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf336-765f-4d49-800b-3d9c0d0a70f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.584598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf336-765f-4d49-800b-3d9c0d0a70f0 admin admin] PATH_INFO: `/v3/projects/4d6e971f1cab47f2955d37873dfcceab/users/6bdf63e5bfd2476f856900d5b941740b/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.597449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.598467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.599287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.600424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.605000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:01.622017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf336-765f-4d49-800b-3d9c0d0a70f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6e971f1cab47f2955d37873dfcceab, user_id=6bdf63e5bfd2476f856900d5b941740b, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.624387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf336-765f-4d49-800b-3d9c0d0a70f0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.641445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:01.656523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 967/1945] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:01 2022] PUT /identity/v3/projects/4d6e971f1cab47f2955d37873dfcceab/users/6bdf63e5bfd2476f856900d5b941740b/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:01.669230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.669921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.670338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.721254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] MFA Rules not processed for user `6bdf63e5bfd2476f856900d5b941740b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:01.722723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:01.739070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.739994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.758646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 979/1946] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:01.765719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:01.806051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 968/1947] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:01.811262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:01.825554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa4209-9884-4e46-a678-910d9e8baf14 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:01.933878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:01.936481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:01.936864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:01.937299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:01.938084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:01.940042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:01.988480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 980/1948] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:01 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:01.992275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.009235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955dac33-1307-4e7e-948e-4128661a3c19 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.127275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.128016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.128292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.128569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.129536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:02.131558 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:02.171784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 969/1949] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:01 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:02.173359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.192920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed32bb-adcc-47ef-b9b8-1792f10f1f87 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.201977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.202686 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.204338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.204498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.205351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:02.210438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:02.231754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 981/1950] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:02.235995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.252185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701edc7-630f-49ed-a3af-e40df52a5966 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.263725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.265027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.266255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.266552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] PATH_INFO: `/v3/projects/235a790158624675b5967dd0f7ad136f/users/cbc58e6f0b2d46789a3576df68489642/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.286944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=235a790158624675b5967dd0f7ad136f, user_id=cbc58e6f0b2d46789a3576df68489642, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:02.290405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:02.313951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.315308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 970/1951] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:02 2022] PUT /identity/v3/projects/235a790158624675b5967dd0f7ad136f/users/cbc58e6f0b2d46789a3576df68489642/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:02.328068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895799e-8f18-452f-8b8b-d216abeb4e26 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.409297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b748b-093c-4f89-ae29-84db2d5e2026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.412700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b748b-093c-4f89-ae29-84db2d5e2026 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.413292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b748b-093c-4f89-ae29-84db2d5e2026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.413673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b748b-093c-4f89-ae29-84db2d5e2026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.414489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b748b-093c-4f89-ae29-84db2d5e2026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:02.416193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b748b-093c-4f89-ae29-84db2d5e2026 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:02.428708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 982/1952] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:02.432763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.448401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb51692-c2ec-4327-810b-dbf6b6daeeb0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.463776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.464917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.465523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.475376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] PATH_INFO: `/v3/projects/235a790158624675b5967dd0f7ad136f/users/cbc58e6f0b2d46789a3576df68489642/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.481556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=235a790158624675b5967dd0f7ad136f, user_id=cbc58e6f0b2d46789a3576df68489642, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:02.485258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:02.518621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.518910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.519193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.521621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 971/1953] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:02 2022] PUT /identity/v3/projects/235a790158624675b5967dd0f7ad136f/users/cbc58e6f0b2d46789a3576df68489642/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:02.556648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] MFA Rules not processed for user `cbc58e6f0b2d46789a3576df68489642`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:02.558448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:02.591293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:02.679490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 983/1954] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:02.695141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.787798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927ec1c-2d7d-4455-9330-06ad91b9a842 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.792678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.793422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.793963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.794758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:02.806563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:02.807993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:02.818709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 972/1955] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:02.931442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:02.932997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:02.951447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc98a3-7114-4ca7-930c-6eb7f25ee8fb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:02.955891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:02.957106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:02.958727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:02.959166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:03.020133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:03.021716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:03.031500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 984/1956] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:08.695580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:08.697312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:08.724187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:08.726573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:08.743540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:08.792379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e3924-06fe-45c3-9ee3-1940bcc3a26a service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:08.797312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:08.797931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315befb-269b-4e70-9a28-87ac172357a4 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:08.798214 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:08.798492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:08.798769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] PATH_INFO: `/v3/users/849564e986124bbc9185ebe01bf5403c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:08.800022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849564e986124bbc9185ebe01bf5403c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:08.802191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:08.803942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:08.804900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:08.805252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:08.805627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] PATH_INFO: `/v3/users/98df8d75ee16460e9ff2bb415c8782c8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:08.806903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98df8d75ee16460e9ff2bb415c8782c8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:08.808914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:08.857218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849564e986124bbc9185ebe01bf5403c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.863108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98df8d75ee16460e9ff2bb415c8782c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.872910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98df8d75ee16460e9ff2bb415c8782c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.873144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849564e986124bbc9185ebe01bf5403c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.873866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849564e986124bbc9185ebe01bf5403c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.874280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98df8d75ee16460e9ff2bb415c8782c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.874487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849564e986124bbc9185ebe01bf5403c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.874894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98df8d75ee16460e9ff2bb415c8782c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.876918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:08.883410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8009f10a-b900-48dd-89d3-383bb5535540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849564e986124bbc9185ebe01bf5403c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.893774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c193e7f6-4086-48c5-8811-f7ce84d48cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98df8d75ee16460e9ff2bb415c8782c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:08.897306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 973/1957] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:08 2022] DELETE /identity/v3/users/849564e986124bbc9185ebe01bf5403c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:08.903141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:08.903457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:08.903899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:08.910747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 985/1958] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:08 2022] DELETE /identity/v3/users/98df8d75ee16460e9ff2bb415c8782c8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:08.915089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:08.915371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:08.915646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:08.923339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:08.924549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:08.933248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:08.934528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:08.991756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 974/1959] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:08.999267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.001043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.009106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 986/1960] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:09.016578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.018131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.068644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922fee-1f92-4cb8-94e8-17bed350a1ea None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.072382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.073044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.073329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.073708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.074697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d109034-e52a-4e8f-8c9d-38e47ddf5f2c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.078554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c25bf9-0f54-4176-a1ea-2cb0cdb06ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.079219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25bf9-0f54-4176-a1ea-2cb0cdb06ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.079496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25bf9-0f54-4176-a1ea-2cb0cdb06ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.079767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25bf9-0f54-4176-a1ea-2cb0cdb06ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.081832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:09.085638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.088749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.090462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c25bf9-0f54-4176-a1ea-2cb0cdb06ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.092273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c25bf9-0f54-4176-a1ea-2cb0cdb06ffc service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.101295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 975/1961] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:09.102239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 987/1962] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:09.140837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.142573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.157044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98aa182-dd17-45f5-a362-b30f2cced2a7 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.162493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.163554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.163959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.164365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] PATH_INFO: `/v3/users/6bdf63e5bfd2476f856900d5b941740b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.165735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bdf63e5bfd2476f856900d5b941740b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.167985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.221894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bdf63e5bfd2476f856900d5b941740b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.247812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bdf63e5bfd2476f856900d5b941740b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.249446 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bdf63e5bfd2476f856900d5b941740b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.250360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bdf63e5bfd2476f856900d5b941740b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.258628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bdf63e5bfd2476f856900d5b941740b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.275423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 976/1963] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:09 2022] DELETE /identity/v3/users/6bdf63e5bfd2476f856900d5b941740b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:09.280416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.280701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.280973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.301071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:09.302475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.378568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 988/1964] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:09.399734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.403479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.478447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46918d-11ca-4fb9-9a49-b8bc42900e1d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.482886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.483551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.483838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.484119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.495803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.497275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.510173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 977/1965] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:09.521328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.522901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.557785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad4f6b-7701-4634-9610-5bb5aa5dc661 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.570661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.572492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.581142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.583202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.583485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.585036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] PATH_INFO: `/v3/projects/2e148a0e86cb4229a06bf3afcb314db1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.588145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e148a0e86cb4229a06bf3afcb314db1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.595441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.599109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919dc2d-7746-42f7-bcf8-0b95232158cd service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.619702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.620607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.620897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.621177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] PATH_INFO: `/v3/projects/5a350c001c0649b3adf4b1dcbae0a717` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.626785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a350c001c0649b3adf4b1dcbae0a717)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.626944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:09.632238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.703166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e148a0e86cb4229a06bf3afcb314db1', 'request_id': 'req-11d20d78-4a50-469a-9275-41a5fdf1cd6c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.717548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e148a0e86cb4229a06bf3afcb314db1', 'request_id': 'req-11d20d78-4a50-469a-9275-41a5fdf1cd6c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.719068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a350c001c0649b3adf4b1dcbae0a717', 'request_id': 'req-ef9b5205-9619-464a-b888-6990986fcf34'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.736953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e148a0e86cb4229a06bf3afcb314db1', 'request_id': 'req-11d20d78-4a50-469a-9275-41a5fdf1cd6c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.741089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a350c001c0649b3adf4b1dcbae0a717', 'request_id': 'req-ef9b5205-9619-464a-b888-6990986fcf34'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.754740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a350c001c0649b3adf4b1dcbae0a717', 'request_id': 'req-ef9b5205-9619-464a-b888-6990986fcf34'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.767237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:09.771043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] The token cache is being invalidate because project 2e148a0e86cb4229a06bf3afcb314db1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:09.771043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.771687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.775474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 989/1966] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:09 2022] DELETE /identity/v3/projects/2e148a0e86cb4229a06bf3afcb314db1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:09.778247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] The token cache is being invalidate because project 5a350c001c0649b3adf4b1dcbae0a717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:09.778548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.779319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.779542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.781259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.785135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 978/1967] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:09 2022] DELETE /identity/v3/projects/5a350c001c0649b3adf4b1dcbae0a717 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:09.786152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:09.787983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:09.873074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5205-9619-464a-b888-6990986fcf34 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.878408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.878995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.879539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.880237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] PATH_INFO: `/v3/users/9efa9fe381f7478a9ef463fe2fe6cf02` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.881123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9efa9fe381f7478a9ef463fe2fe6cf02)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.883293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.900367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d20d78-4a50-469a-9275-41a5fdf1cd6c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:09.906725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:09.908447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:09.908740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:09.909017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] PATH_INFO: `/v3/users/511d12b1ba5c441e95bda62365a671ec` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:09.910324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=511d12b1ba5c441e95bda62365a671ec)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:09.912285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:09.958273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9efa9fe381f7478a9ef463fe2fe6cf02'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.976915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9efa9fe381f7478a9ef463fe2fe6cf02'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.978602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9efa9fe381f7478a9ef463fe2fe6cf02'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.980264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9efa9fe381f7478a9ef463fe2fe6cf02'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.991854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9efa9fe381f7478a9ef463fe2fe6cf02'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:09.992697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '511d12b1ba5c441e95bda62365a671ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.004242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '511d12b1ba5c441e95bda62365a671ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.005298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '511d12b1ba5c441e95bda62365a671ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.007009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '511d12b1ba5c441e95bda62365a671ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.024671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 979/1968] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:09 2022] DELETE /identity/v3/users/9efa9fe381f7478a9ef463fe2fe6cf02 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:10.025162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '511d12b1ba5c441e95bda62365a671ec'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.026150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:10.044277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 990/1969] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:09 2022] DELETE /identity/v3/users/511d12b1ba5c441e95bda62365a671ec => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:10.044425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:10.139626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13365d-4ce7-427d-9b74-b05803564f84 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:10.160117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:10.219107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:10.220109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:10.220396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:10.220925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] PATH_INFO: `/v3/projects/4d6e971f1cab47f2955d37873dfcceab` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:10.222299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6e971f1cab47f2955d37873dfcceab)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:10.224395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:10.287299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6e971f1cab47f2955d37873dfcceab', 'request_id': 'req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.298724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6e971f1cab47f2955d37873dfcceab', 'request_id': 'req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.306900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6e971f1cab47f2955d37873dfcceab', 'request_id': 'req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.319046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] The token cache is being invalidate because project 4d6e971f1cab47f2955d37873dfcceab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:10.319046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.319900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.321786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 980/1970] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:10 2022] DELETE /identity/v3/projects/4d6e971f1cab47f2955d37873dfcceab => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:10.325777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:10.327393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:10.343387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693c35-df72-422d-b926-04248c41ce64 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:10.365193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:10.419878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:10.420975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:10.421258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:10.421567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] PATH_INFO: `/v3/users/cbc58e6f0b2d46789a3576df68489642` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:10.422813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc58e6f0b2d46789a3576df68489642)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:10.424753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:10.481948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc58e6f0b2d46789a3576df68489642'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.491205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc58e6f0b2d46789a3576df68489642'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.491916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc58e6f0b2d46789a3576df68489642'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.492543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc58e6f0b2d46789a3576df68489642'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.509823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc58e6f0b2d46789a3576df68489642'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.532520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 991/1971] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:10 2022] DELETE /identity/v3/users/cbc58e6f0b2d46789a3576df68489642 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:10.627203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:10.628809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:10.638910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:10.640973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:10.651400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a056b0-4d78-4b28-9fa5-7023f1da0bd3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:10.729553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:10.730750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:10.731154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:10.731527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] PATH_INFO: `/v3/projects/cb362da3e854408c9877faaf4154b4e2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:10.734469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb362da3e854408c9877faaf4154b4e2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:10.747925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:10.764421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab7965-f9f1-4c5f-993a-ad8a86d59fe3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:10.770079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:10.771519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:10.771519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:10.771661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] PATH_INFO: `/v3/projects/acf3d086c6d04304b7c487f18e77dd46` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:10.773107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf3d086c6d04304b7c487f18e77dd46)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:10.775400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:10.834389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:10.841673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb362da3e854408c9877faaf4154b4e2', 'request_id': 'req-6d75ec23-af15-4554-9a2c-ed9e790bba51'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.845474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf3d086c6d04304b7c487f18e77dd46', 'request_id': 'req-14b6192f-47fe-4f67-83f3-7a2311504685'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.852242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:10.859835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf3d086c6d04304b7c487f18e77dd46', 'request_id': 'req-14b6192f-47fe-4f67-83f3-7a2311504685'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.863988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb362da3e854408c9877faaf4154b4e2', 'request_id': 'req-6d75ec23-af15-4554-9a2c-ed9e790bba51'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.875788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb362da3e854408c9877faaf4154b4e2', 'request_id': 'req-6d75ec23-af15-4554-9a2c-ed9e790bba51'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.876428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf3d086c6d04304b7c487f18e77dd46', 'request_id': 'req-14b6192f-47fe-4f67-83f3-7a2311504685'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.891273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] The token cache is being invalidate because project cb362da3e854408c9877faaf4154b4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:10.891666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.892604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6d75ec23-af15-4554-9a2c-ed9e790bba51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.898129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 981/1972] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:10 2022] DELETE /identity/v3/projects/cb362da3e854408c9877faaf4154b4e2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:10.899742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d2b7-67fe-4985-880c-8e978df91189 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:10.900233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d2b7-67fe-4985-880c-8e978df91189 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:10.900849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d2b7-67fe-4985-880c-8e978df91189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:10.908634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] The token cache is being invalidate because project acf3d086c6d04304b7c487f18e77dd46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:10.911200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.912237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-14b6192f-47fe-4f67-83f3-7a2311504685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:10.921071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 992/1973] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:10 2022] DELETE /identity/v3/projects/acf3d086c6d04304b7c487f18e77dd46 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:10.922472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:10.922775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:10.923213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:10.933236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d4e3d2b7-67fe-4985-880c-8e978df91189 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:10.934290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d4e3d2b7-67fe-4985-880c-8e978df91189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:10.944991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:10.946690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.039380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 982/1974] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.041319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.042009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.042243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.068036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 993/1975] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.069395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.071237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.073386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:11.074588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.091135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d2386-dd97-49f2-a2d4-2ea315f0db45 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.096812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 983/1976] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.097393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.097948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.098747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.099040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.099320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] PATH_INFO: `/v3/projects/235a790158624675b5967dd0f7ad136f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.099835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.100558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] RBAC: Authorizing `identity:delete_project(project_id=235a790158624675b5967dd0f7ad136f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.102755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.115702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617274f-e3ac-4851-9248-11a78e29b577 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.121387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.122377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.122800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.123130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] PATH_INFO: `/v3/users/98898eedb0d843c18aa24fb7fd60d52e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.124362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=98898eedb0d843c18aa24fb7fd60d52e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.126868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.178431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235a790158624675b5967dd0f7ad136f', 'request_id': 'req-8294725a-8e47-496b-b126-add9417b341c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.190807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235a790158624675b5967dd0f7ad136f', 'request_id': 'req-8294725a-8e47-496b-b126-add9417b341c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.197354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235a790158624675b5967dd0f7ad136f', 'request_id': 'req-8294725a-8e47-496b-b126-add9417b341c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.197687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98898eedb0d843c18aa24fb7fd60d52e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.210663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] The token cache is being invalidate because project 235a790158624675b5967dd0f7ad136f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:11.210663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.210663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8294725a-8e47-496b-b126-add9417b341c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.211862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 994/1977] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:11 2022] DELETE /identity/v3/projects/235a790158624675b5967dd0f7ad136f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:11.216586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478812c-80c1-4eed-b74f-f0ec06b006fc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.216753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478812c-80c1-4eed-b74f-f0ec06b006fc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.217118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478812c-80c1-4eed-b74f-f0ec06b006fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.217704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98898eedb0d843c18aa24fb7fd60d52e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.218925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98898eedb0d843c18aa24fb7fd60d52e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.219680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98898eedb0d843c18aa24fb7fd60d52e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.229503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98898eedb0d843c18aa24fb7fd60d52e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.237934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1478812c-80c1-4eed-b74f-f0ec06b006fc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:11.239142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1478812c-80c1-4eed-b74f-f0ec06b006fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.258713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 984/1978] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:11 2022] DELETE /identity/v3/users/98898eedb0d843c18aa24fb7fd60d52e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:11.259662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.262468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.324189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:11.339658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:11.347445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 995/1979] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.349134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187de08-ace2-4cd9-83e3-4880cd2d13a6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.349452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187de08-ace2-4cd9-83e3-4880cd2d13a6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.349738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187de08-ace2-4cd9-83e3-4880cd2d13a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.367001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3dc57-96bf-4319-a3ca-5cb0ddb522fa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.374148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.374670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.375045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.375768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8187de08-ace2-4cd9-83e3-4880cd2d13a6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:11.375883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.376211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:11.376587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.376692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8187de08-ace2-4cd9-83e3-4880cd2d13a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.378316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.386698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 996/1980] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.387921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828c92-358f-4209-8b88-9ece2b91ab83 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.390061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828c92-358f-4209-8b88-9ece2b91ab83 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.390061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10828c92-358f-4209-8b88-9ece2b91ab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.401266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 985/1981] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:11.401266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.412586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257fd15-9ead-49e5-8330-6e9190d4bc4c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.417455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-10828c92-358f-4209-8b88-9ece2b91ab83 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:11.417455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-10828c92-358f-4209-8b88-9ece2b91ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.420614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.421302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.421627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.421931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.422690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:11.423004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.424619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.425746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 997/1982] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.429908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.430836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.430836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.445753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 986/1983] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:11.445753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.448835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:11.450066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.452935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592b570-d441-4a04-be65-d32031188418 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.457471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.458116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.461485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.461612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 998/1984] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:11.461693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.461784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.461859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.461928 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.461998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.471969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141592e-81bc-4c15-af03-1ce108e4d753 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.476879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.477821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.478105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.478421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] PATH_INFO: `/v3/users/6a7c30c4b8a948e99694fee159f15aa3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.479780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a7c30c4b8a948e99694fee159f15aa3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.481895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.490735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 987/1985] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:11.490735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.490735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.530872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a7c30c4b8a948e99694fee159f15aa3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.538846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a7c30c4b8a948e99694fee159f15aa3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.539531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a7c30c4b8a948e99694fee159f15aa3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.540140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a7c30c4b8a948e99694fee159f15aa3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.551994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d82b23-80ad-43ad-a0d5-ce6c402de27a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.553099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a7c30c4b8a948e99694fee159f15aa3'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.555733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.556389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.556678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.556957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.558913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.566488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 999/1986] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:11 2022] DELETE /identity/v3/users/6a7c30c4b8a948e99694fee159f15aa3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:11.567267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.568644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.638659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.640605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.642724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926092c2-05a8-42c4-ab3d-4554c91cea8e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.647375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.648041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.648318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.648639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.649338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.651263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.653608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 988/1987] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:11.655084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.656853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.671960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edc761-4508-4cab-83f6-14c893bb1275 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.677827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.678495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.678829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.679210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.679828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:11.680130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.681820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.682354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1000/1988] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:11.683210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.685859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.696900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 989/1989] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:11.698024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.711352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196aa45-881d-4d9f-90ef-0edd85a3a4b9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.730444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:11.757582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:11.780527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.781599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.782031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.782450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.783761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.785968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.787061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da629f1-e471-4d27-bd20-67762926dc8a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.795040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.795040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.795485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.795922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.797158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.800047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.811498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 990/1990] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:11.812492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.825812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d74f7-2de9-4cde-97b8-540ae7fa1745 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.833676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.833891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1001/1991] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/users => generated 282 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:11.833991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.834079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.834164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.834164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.834164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.835026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.835835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:11.848644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f66b2e-7b00-4908-8ffc-ec499459e77a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.856220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:11.857321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:11.857815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:11.858239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] PATH_INFO: `/v3/projects/641fe924d4684aecbdcea861184019fe` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:11.859634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=641fe924d4684aecbdcea861184019fe)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:11.862773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:11.882107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 991/1992] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:11 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:11.883117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.921157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641fe924d4684aecbdcea861184019fe', 'request_id': 'req-02c6d379-a53f-4e42-addd-f2c495e17369'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.931333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641fe924d4684aecbdcea861184019fe', 'request_id': 'req-02c6d379-a53f-4e42-addd-f2c495e17369'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.941350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641fe924d4684aecbdcea861184019fe', 'request_id': 'req-02c6d379-a53f-4e42-addd-f2c495e17369'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.955936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567aff-acd4-4e8a-828a-647bc5aae710 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:11.958453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] The token cache is being invalidate because project 641fe924d4684aecbdcea861184019fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:11.958925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.959616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:11.961831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1002/1993] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:11 2022] DELETE /identity/v3/projects/641fe924d4684aecbdcea861184019fe => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:11.962850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:11.964254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.025488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.026536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.027008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.027357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.028162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.030470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.046260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 992/1994] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:12.046260 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.048468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.052517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6d379-a53f-4e42-addd-f2c495e17369 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.057775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.058497 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.059491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.059772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.061102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.062341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.063609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495b931-41f9-4acf-8865-d79e6ae07bd9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.069452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.070770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.071053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.071491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.072194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.077095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1003/1995] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:12.077257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.077338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.079232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.092799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00023030-87c3-4fbe-a65e-74bcee9911f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.097681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.098568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.098851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.099273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] PATH_INFO: `/v3/users/80ebbc6807f1425380c56322b09d17f7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.103801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ebbc6807f1425380c56322b09d17f7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.103949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 993/1996] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:12 2022] POST /identity/v3/users => generated 290 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:12.103949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.104092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.106270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.122059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc2b8c-093d-464a-bc88-da99edbfc773 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.129454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.130716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.130716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.130716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.131623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.133373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.144893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 994/1997] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:12.145828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.158225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3de83-c7d3-47e5-9a7f-41dc5e1d3a9a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.162268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:12.164002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.164985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.165273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.165595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.166775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.168590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.179806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ebbc6807f1425380c56322b09d17f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.181282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 995/1998] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:12.182307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.195525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b22c7-b85f-4bd5-ae14-d372a77edd9d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.195842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ebbc6807f1425380c56322b09d17f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.196525 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ebbc6807f1425380c56322b09d17f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.197217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ebbc6807f1425380c56322b09d17f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.200986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.201710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.201936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.202237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.202940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.204799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.212190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ebbc6807f1425380c56322b09d17f7'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.217451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 996/1999] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:12.217451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.220482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.228419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:12.228997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1004/2000] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:12 2022] DELETE /identity/v3/users/80ebbc6807f1425380c56322b09d17f7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.233120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.234833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.236794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef10dca-ad8d-47ca-8672-1bed6afb5624 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.313526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ffe22-247a-42c5-a630-74ca277700af admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.318283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.319181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.319468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.319734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] PATH_INFO: `/v3/projects/7613728bfab941b8ad8a62b74d8edbcd/users/3fefa9b5c4064adc90456e5706108343/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.323443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.324401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.324683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.324959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] PATH_INFO: `/v3/projects/894127095d6445f5a2eafc9906ce5cf7/users/fafefcbb62bf4284a8a9da5d6c0a4609/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.338574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7613728bfab941b8ad8a62b74d8edbcd, user_id=3fefa9b5c4064adc90456e5706108343, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.340858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894127095d6445f5a2eafc9906ce5cf7, user_id=fafefcbb62bf4284a8a9da5d6c0a4609, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.341381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.344369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.359427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 997/2001] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:12 2022] PUT /identity/v3/projects/894127095d6445f5a2eafc9906ce5cf7/users/fafefcbb62bf4284a8a9da5d6c0a4609/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.360125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.360481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1005/2002] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:12 2022] PUT /identity/v3/projects/7613728bfab941b8ad8a62b74d8edbcd/users/3fefa9b5c4064adc90456e5706108343/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.361604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.362927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.363208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.457064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd070c5-14c2-4460-ae87-1e0192c05e80 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.462189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.465990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.466566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.466923 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] PATH_INFO: `/v3/projects/01377371ddd74cb990b95aec171e2f14` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.468311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=01377371ddd74cb990b95aec171e2f14)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.470510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.474745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413a47e-975e-4b9e-811e-b6ad936b8d3f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.481601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a4a1d-1370-4a47-8b53-d958ad4ae576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.482547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a4a1d-1370-4a47-8b53-d958ad4ae576 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.482900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a4a1d-1370-4a47-8b53-d958ad4ae576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.483183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a4a1d-1370-4a47-8b53-d958ad4ae576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.484609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a4a1d-1370-4a47-8b53-d958ad4ae576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.486284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a4a1d-1370-4a47-8b53-d958ad4ae576 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.499762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1006/2003] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:12.503042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.503394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.503702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.539956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01377371ddd74cb990b95aec171e2f14', 'request_id': 'req-1b513544-7280-41ab-913d-5a868e01f4ed'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.540166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] MFA Rules not processed for user `fafefcbb62bf4284a8a9da5d6c0a4609`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:12.541813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.551544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01377371ddd74cb990b95aec171e2f14', 'request_id': 'req-1b513544-7280-41ab-913d-5a868e01f4ed'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.559439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01377371ddd74cb990b95aec171e2f14', 'request_id': 'req-1b513544-7280-41ab-913d-5a868e01f4ed'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.565253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:12.571597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] The token cache is being invalidate because project 01377371ddd74cb990b95aec171e2f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:12.572136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.572651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1b513544-7280-41ab-913d-5a868e01f4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.574443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 998/2004] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:12 2022] DELETE /identity/v3/projects/01377371ddd74cb990b95aec171e2f14 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.575827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.576187 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.576682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.604982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] MFA Rules not processed for user `3fefa9b5c4064adc90456e5706108343`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:12.606215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.610255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:12.627038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1007/2005] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:12.627972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.629272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.674684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 999/2006] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:12.675659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.676922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.702476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92170ab-a9cc-485d-9d67-efdd8d44bf20 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.707132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.708012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.708292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.708568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] PATH_INFO: `/v3/projects/70fbb1ed0fd3462db16a7fbd4571722b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.710050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fbb1ed0fd3462db16a7fbd4571722b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.712116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.743361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78e5ea-c492-42a1-9a10-d87a15a3f0ad None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.748109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.749001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.749280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.749582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] PATH_INFO: `/v3/projects/9835a5dd19014c4392216eebeed00a86/users/a89b40bedab34463992bc4248826996c/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.759526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fbb1ed0fd3462db16a7fbd4571722b', 'request_id': 'req-21f70057-f9c8-47db-ac8d-f35adb9944bd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.764804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9835a5dd19014c4392216eebeed00a86, user_id=a89b40bedab34463992bc4248826996c, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.767846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.771003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fbb1ed0fd3462db16a7fbd4571722b', 'request_id': 'req-21f70057-f9c8-47db-ac8d-f35adb9944bd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.777734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fbb1ed0fd3462db16a7fbd4571722b', 'request_id': 'req-21f70057-f9c8-47db-ac8d-f35adb9944bd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.782946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1000/2007] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:12 2022] PUT /identity/v3/projects/9835a5dd19014c4392216eebeed00a86/users/a89b40bedab34463992bc4248826996c/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.783772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.785336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.790073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] The token cache is being invalidate because project 70fbb1ed0fd3462db16a7fbd4571722b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:12.790889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.792041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.793921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1008/2008] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:12 2022] DELETE /identity/v3/projects/70fbb1ed0fd3462db16a7fbd4571722b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.794914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.796298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.851653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f70057-f9c8-47db-ac8d-f35adb9944bd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.851841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432702b-ff65-415a-9125-fd4a8e14d235 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:12.855077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:12.856566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.857772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.858117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.858397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] PATH_INFO: `/v3/users/bceb6c43b2774e0aac9344d42de01efe` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.859611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bceb6c43b2774e0aac9344d42de01efe)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.861712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.897685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:12.903625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:12.904566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.905180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.905180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:12.907544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.913288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bceb6c43b2774e0aac9344d42de01efe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.925776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bceb6c43b2774e0aac9344d42de01efe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.926468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bceb6c43b2774e0aac9344d42de01efe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.927079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bceb6c43b2774e0aac9344d42de01efe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.934220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bceb6c43b2774e0aac9344d42de01efe'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:12.947183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1001/2009] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:12 2022] DELETE /identity/v3/users/bceb6c43b2774e0aac9344d42de01efe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:12.951015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:12.952366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:12.967365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:12.968829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5617b-be50-46ea-9b94-484e694ec647 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:12.979798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1009/2010] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:12.981177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:12.981509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:12.981789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.013003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] MFA Rules not processed for user `a89b40bedab34463992bc4248826996c`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:13.014874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.039853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc47edb-e309-48f6-9340-d4f6dcaf6038 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.046329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528f3e-6544-46cc-9b91-43c1e5a1f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.047004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528f3e-6544-46cc-9b91-43c1e5a1f41e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.047285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528f3e-6544-46cc-9b91-43c1e5a1f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.047560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528f3e-6544-46cc-9b91-43c1e5a1f41e admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.049176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca528f3e-6544-46cc-9b91-43c1e5a1f41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.051907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca528f3e-6544-46cc-9b91-43c1e5a1f41e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.092797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1002/2011] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:12 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:13.094493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.094971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.095607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.106639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1010/2012] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:13.106639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.119590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:13.121170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.137097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb722549-51c0-4359-91a8-4816aab04972 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.199845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1003/2013] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:13.201026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.202907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.219556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aebb31-e116-46a6-b5d7-150efbab627a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.220232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:13.226235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5ea9-fea3-4866-8fa1-74ca1e9f9cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.227181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5ea9-fea3-4866-8fa1-74ca1e9f9cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.227617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5ea9-fea3-4866-8fa1-74ca1e9f9cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.227889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5ea9-fea3-4866-8fa1-74ca1e9f9cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.228590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5ea9-fea3-4866-8fa1-74ca1e9f9cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.230546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5ea9-fea3-4866-8fa1-74ca1e9f9cf5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.235633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.239991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.241127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.245551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.246294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.251031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.276407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1004/2014] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:13.278083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.278509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.278880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.282901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1011/2015] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:13.284046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.286801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.304871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:13.307001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.312318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4af515-6cbf-466f-8abe-086d6fd1d260 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.313400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:13.414180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1005/2016] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:13.414180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.414447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.414447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.414447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.414447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] PATH_INFO: `/v3/projects/5ec84ae9d38143f98e69ba431ace94fa` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.417185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec84ae9d38143f98e69ba431ace94fa)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.418080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.426418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a985f-2b29-43eb-91a4-b92130dad38c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.434796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.435854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.436148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.436428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.437277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.439658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.468168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1006/2017] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/users => generated 297 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:13.469700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.475217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec84ae9d38143f98e69ba431ace94fa', 'request_id': 'req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:13.480927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a29eb-2588-44e1-9990-50c06c4e72bd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.483443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec84ae9d38143f98e69ba431ace94fa', 'request_id': 'req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:13.489674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec84ae9d38143f98e69ba431ace94fa', 'request_id': 'req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:13.502897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] The token cache is being invalidate because project 5ec84ae9d38143f98e69ba431ace94fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:13.503186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:13.503854 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:13.505542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1012/2018] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:13 2022] DELETE /identity/v3/projects/5ec84ae9d38143f98e69ba431ace94fa => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:13.506311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.507589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.541843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.542554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.542835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.543147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.543877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.545967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.556345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1007/2019] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:13.557143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.561133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.570451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e7fc4-d21b-4e8b-b461-fbca2ff6f650 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.577033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b11ad-fd01-4835-84f6-a26aa97c75f5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.578073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.578865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.579088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.579421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.580146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:13.580532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.582238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be27ebc-63e0-45ff-b8eb-03143fddffcb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.583121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.583794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.584084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.584381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.585103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.586765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.600349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1013/2020] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:13.600530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1008/2021] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:13.600612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.600612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.600774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.600849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.602350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.614486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:13.617912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcda943-5518-429c-bdb1-96a04e445701 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.623635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.624686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.625084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.625441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] PATH_INFO: `/v3/projects/5a6edce2b99a4b8f929d0cd173328ccc/users/944f77f3811d4203b62b1d75e64f4a3b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.630091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:13.631562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.641204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1014/2022] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:13.642567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.652701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6edce2b99a4b8f929d0cd173328ccc, user_id=944f77f3811d4203b62b1d75e64f4a3b, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.653343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb59b7-184e-47c9-aab7-d0e3b6f970a4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.655834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.658258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618f3ef-c4d2-4fa6-a822-d9918515001a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.658920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618f3ef-c4d2-4fa6-a822-d9918515001a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.659197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618f3ef-c4d2-4fa6-a822-d9918515001a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.659767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618f3ef-c4d2-4fa6-a822-d9918515001a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.660242 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618f3ef-c4d2-4fa6-a822-d9918515001a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.662277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618f3ef-c4d2-4fa6-a822-d9918515001a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.673338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1009/2023] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:13 2022] PUT /identity/v3/projects/5a6edce2b99a4b8f929d0cd173328ccc/users/944f77f3811d4203b62b1d75e64f4a3b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:13.674824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.708551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1015/2024] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:13.708551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.708551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.708551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.728120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:13.729346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:13.747626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:13.756037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f49b4a-60a0-4888-b943-3789fb668494 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.793558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1016/2025] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:13.794615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.806287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b2af7-bff0-4e20-8943-297646cc3542 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.811937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.812606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.812892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.813165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.813945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.815906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.820942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.822286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.822589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.822860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] PATH_INFO: `/v3/projects/f35ac0c0323849efb92cc561439823f5/users/181148aa7c0d4fc697e2adfab1fbe887/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.826220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1017/2026] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:13.827135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.837557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dd211-a098-4861-b033-d5d4b6edbe18 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.838209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35ac0c0323849efb92cc561439823f5, user_id=181148aa7c0d4fc697e2adfab1fbe887, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.841063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.842657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.843317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.843604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.843885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.844893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.846709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.855767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1010/2027] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:13 2022] PUT /identity/v3/projects/f35ac0c0323849efb92cc561439823f5/users/181148aa7c0d4fc697e2adfab1fbe887/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:13.856739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.870667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1018/2028] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:13 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:13.871590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.885133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41283648-d495-4c63-a33f-29200fd408f7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.925097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588e00d-ca99-45ef-8067-1601e91725b0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.930739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.931829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.932122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.932556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.933328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:13.933803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.935367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.947205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:13.947751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1011/2029] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:13.948091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:13.948378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:13.948582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:13.948665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] PATH_INFO: `/v3/projects/5a6edce2b99a4b8f929d0cd173328ccc/users/944f77f3811d4203b62b1d75e64f4a3b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:13.950744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6edce2b99a4b8f929d0cd173328ccc, user_id=944f77f3811d4203b62b1d75e64f4a3b, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:13.953438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:13.958692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951ee68-2bf2-4d6d-b955-3a1f35e74994 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:13.964025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1019/2030] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:13 2022] PUT /identity/v3/projects/5a6edce2b99a4b8f929d0cd173328ccc/users/944f77f3811d4203b62b1d75e64f4a3b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:13.964983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.014055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:14.032996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30607d8a-4f43-41d7-b0c0-e7d662b1f4fe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.036040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.036721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.037013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.037289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.038053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.038685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea643bec-a1ce-4f3a-befc-d67bea00eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.039361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643bec-a1ce-4f3a-befc-d67bea00eebb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.039647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643bec-a1ce-4f3a-befc-d67bea00eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.039766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.039943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea643bec-a1ce-4f3a-befc-d67bea00eebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.040653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643bec-a1ce-4f3a-befc-d67bea00eebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.042586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea643bec-a1ce-4f3a-befc-d67bea00eebb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.055477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1012/2031] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:14.055662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1020/2032] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:14.055785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.055878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.055878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.055878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.067384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed043191-8aed-4497-86e5-9e5029d46800 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.072043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.072701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.072990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.073273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.074010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.076065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.081467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] MFA Rules not processed for user `944f77f3811d4203b62b1d75e64f4a3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:14.082676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.091402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:14.112905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1013/2033] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:14.113823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.157450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1021/2034] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:14.158159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.171938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6f035-a7b6-4c1a-b08d-68f129c5c46b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.183591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0911a8-314a-402f-9187-c0351eecefb6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.188902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.189762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.190044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.190322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.191053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.192790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.203433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1014/2035] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:14.207368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.217666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156d3a9-8aa6-468e-afa9-38d2cc681508 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.222570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.223243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.223528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.223803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.224475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.226430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.228537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.229485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.229765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.230041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] PATH_INFO: `/v3/projects/f35ac0c0323849efb92cc561439823f5/users/181148aa7c0d4fc697e2adfab1fbe887/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.232193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f35ac0c0323849efb92cc561439823f5, user_id=181148aa7c0d4fc697e2adfab1fbe887, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.234846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.246496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1022/2036] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:14 2022] PUT /identity/v3/projects/f35ac0c0323849efb92cc561439823f5/users/181148aa7c0d4fc697e2adfab1fbe887/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:14.247066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.251099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1015/2037] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:14.251099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.252002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.266210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52784a76-a5da-4843-b54a-5ccdcbe5fa3c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.312459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11902803-c0a4-4400-83aa-312be44b5a3c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.316148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cbc9b-a291-4095-a867-0012f5925780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.316796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbc9b-a291-4095-a867-0012f5925780 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.317074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbc9b-a291-4095-a867-0012f5925780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.317345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cbc9b-a291-4095-a867-0012f5925780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.325769 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.326554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:14.326918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.327202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.327845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] PATH_INFO: `/v3/projects/019830178bc945da8d9e6924d1b214a9/users/0d001589bb8a41599db42101b09d9c7a/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.343830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] RBAC: Authorizing `identity:create_grant(project_id=019830178bc945da8d9e6924d1b214a9, user_id=0d001589bb8a41599db42101b09d9c7a, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.348150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.364375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1016/2038] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:14 2022] PUT /identity/v3/projects/019830178bc945da8d9e6924d1b214a9/users/0d001589bb8a41599db42101b09d9c7a/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:14.373594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.382566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cbc9b-a291-4095-a867-0012f5925780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.385777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2cbc9b-a291-4095-a867-0012f5925780 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.398767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1023/2039] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:14.400595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb184-6bb4-43c6-b1a7-a7ee4d7f1377 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.401183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb184-6bb4-43c6-b1a7-a7ee4d7f1377 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.401665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb184-6bb4-43c6-b1a7-a7ee4d7f1377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.438722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-907eb184-6bb4-43c6-b1a7-a7ee4d7f1377 None None] MFA Rules not processed for user `181148aa7c0d4fc697e2adfab1fbe887`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:14.440206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-907eb184-6bb4-43c6-b1a7-a7ee4d7f1377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.456945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:14.467608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa579ab3-c855-45f4-b0f5-8f343a2a1add admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.475353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7289b-2b64-49c0-892e-6c2e8d0ac7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.476473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7289b-2b64-49c0-892e-6c2e8d0ac7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.476811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7289b-2b64-49c0-892e-6c2e8d0ac7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.477276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7289b-2b64-49c0-892e-6c2e8d0ac7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.478115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7289b-2b64-49c0-892e-6c2e8d0ac7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.480219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7289b-2b64-49c0-892e-6c2e8d0ac7ad admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.505229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1017/2040] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:14.506501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.506790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.507064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.523746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1024/2041] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:14.526269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.526269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.526269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.537618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] MFA Rules not processed for user `0d001589bb8a41599db42101b09d9c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:14.538842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.548434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] MFA Rules not processed for user `fafefcbb62bf4284a8a9da5d6c0a4609`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:14.549652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.602594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1025/2042] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:14.603441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.604719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.605613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1018/2043] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:14.606941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.618785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becd1a5-96e6-4368-b5eb-f8464e442d84 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.624193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.625316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.625620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.625894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.627018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.628936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.641258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1019/2044] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:14.642129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.652235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:14.679301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0b4c7-1a13-4167-bfee-76b8a3452560 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.683250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.683910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.684312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.684604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.686665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:14.707789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcccd41-1d59-4ed0-9f6f-69cb8c177ba3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.712316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.712975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.713259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.713581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.723485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.725003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.735754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1020/2045] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:14.736647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.746716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aab223-fd8b-4598-a417-626bca0c8c76 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.750248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.751672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.752356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.753200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.753307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.753614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.754320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.756289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.761798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1026/2046] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:14.772185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.794634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1021/2047] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:14.794937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6393f5-69ee-4582-b33c-e599b8aca1e5 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.802010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:14.868975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc4fd-373e-4bb8-8aed-55ad35e9b6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:14.869856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc4fd-373e-4bb8-8aed-55ad35e9b6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.870142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc4fd-373e-4bb8-8aed-55ad35e9b6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.870471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc4fd-373e-4bb8-8aed-55ad35e9b6ba admin admin] PATH_INFO: `/v3/projects/e1abb28326074db1a13efed4ec68ee34/users/ad82e8d32cab4dcba32edd95a761a0f2/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:14.889021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc4fd-373e-4bb8-8aed-55ad35e9b6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1abb28326074db1a13efed4ec68ee34, user_id=ad82e8d32cab4dcba32edd95a761a0f2, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:14.894191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc4fd-373e-4bb8-8aed-55ad35e9b6ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:14.913397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:14.928057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1027/2048] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:14 2022] PUT /identity/v3/projects/e1abb28326074db1a13efed4ec68ee34/users/ad82e8d32cab4dcba32edd95a761a0f2/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:14.935442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28552047-c4a4-48b0-bddf-b81b303471de admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:14.941324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:14.942036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:14.942404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.004301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] MFA Rules not processed for user `ad82e8d32cab4dcba32edd95a761a0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:15.006549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:15.078334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.079038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.079325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.079605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.080310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.082665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.119698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1022/2049] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/users => generated 288 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:15.123448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.140379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1028/2050] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:15.140579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae3b4a-f545-45bf-a6c4-18f9c194c56a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.141969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.145961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.146681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.146959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.147238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.148213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.149168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:15.150944 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.155641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9052f-69a4-4228-81a0-de851495142b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.167709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1023/2051] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:15.167878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.167878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.167878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.167878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.168836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.170579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.170692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.196442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223a204-d33e-4538-ab7a-1ebdf3102302 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.203105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.203877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.204165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.204496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.205258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.207204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.235845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1024/2052] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:15.240352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.260576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389b613-a955-4258-ba76-e694beda0561 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.261024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1029/2053] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:15.262043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.353069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd028e5-fbac-4fe4-8177-f72e2aff32ce admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.358347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.359099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.359390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.359683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.360417 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.362655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.363272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.364370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.364656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.364967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] PATH_INFO: `/v3/projects/00a115a9b4144454b4c9be79fabdbf22/users/c357ca7929f54efd91082ffeb450d747/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.390539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1030/2054] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:15.391863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.395618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a115a9b4144454b4c9be79fabdbf22, user_id=c357ca7929f54efd91082ffeb450d747, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.398656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.407926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9af36-11f4-48ec-8d54-16e0c597787d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.416782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.417786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.418007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.418307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] PATH_INFO: `/v3/users/a89b40bedab34463992bc4248826996c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.418622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1025/2055] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:15 2022] PUT /identity/v3/projects/00a115a9b4144454b4c9be79fabdbf22/users/c357ca7929f54efd91082ffeb450d747/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:15.419863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.420172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a89b40bedab34463992bc4248826996c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.423330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.490040 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:15.515330 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f8fd-4ff2-4036-9da0-df1d159637f1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.518352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a89b40bedab34463992bc4248826996c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:15.521191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f256ca5-bac7-450d-b492-b4a85f2b4173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.521596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f256ca5-bac7-450d-b492-b4a85f2b4173 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.521875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f256ca5-bac7-450d-b492-b4a85f2b4173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.522257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f256ca5-bac7-450d-b492-b4a85f2b4173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.523974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f256ca5-bac7-450d-b492-b4a85f2b4173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.525663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f256ca5-bac7-450d-b492-b4a85f2b4173 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.528559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a89b40bedab34463992bc4248826996c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:15.529730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a89b40bedab34463992bc4248826996c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:15.530381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a89b40bedab34463992bc4248826996c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:15.538616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1026/2056] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:15.544683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a89b40bedab34463992bc4248826996c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:15.547204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa303410-ce20-4c9a-9146-779239920a26 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.547877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa303410-ce20-4c9a-9146-779239920a26 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.548200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa303410-ce20-4c9a-9146-779239920a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.561153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1031/2057] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:15 2022] DELETE /identity/v3/users/a89b40bedab34463992bc4248826996c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:15.562021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.563491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:15.579604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aa303410-ce20-4c9a-9146-779239920a26 None None] MFA Rules not processed for user `c357ca7929f54efd91082ffeb450d747`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:15.582323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa303410-ce20-4c9a-9146-779239920a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:15.626144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:15.652090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0c69f-e384-4497-b489-5c2895f3c729 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.658098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.659164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.659507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.659844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] PATH_INFO: `/v3/projects/323d66c352254b7a9a033be2c7ea2d23/users/9932ef3f7f874a46b8c07749ce99e095/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.673911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1027/2058] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:15.675743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.676104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.676450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.679289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=323d66c352254b7a9a033be2c7ea2d23, user_id=9932ef3f7f874a46b8c07749ce99e095, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.682814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.699233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:15.700985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1032/2059] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:15 2022] PUT /identity/v3/projects/323d66c352254b7a9a033be2c7ea2d23/users/9932ef3f7f874a46b8c07749ce99e095/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:15.702025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.703862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:15.704585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:15.784445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1028/2060] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:15.785340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.795500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab74f4-64e6-4efb-95f7-a0df1d0d1196 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.796219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec96d40-5f0f-4869-a6b6-1872ca1db4a5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.801513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.802027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.802191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.802528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.802694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.802817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.802975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.803255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.803572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.803948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.805332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.805896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.818297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1029/2061] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:15.819174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.822525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:15.840379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1033/2062] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:15.841470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.894856 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461526c3-a65e-44a4-8a32-c2e2dd6f7c9e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.916047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:15.917024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730b675-c924-426d-be10-2293f6078be5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:15.921874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.922856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.923137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.923406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] PATH_INFO: `/v3/projects/323d66c352254b7a9a033be2c7ea2d23/users/9932ef3f7f874a46b8c07749ce99e095/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.925560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=323d66c352254b7a9a033be2c7ea2d23, user_id=9932ef3f7f874a46b8c07749ce99e095, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.928146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.940386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1034/2063] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:15 2022] PUT /identity/v3/projects/323d66c352254b7a9a033be2c7ea2d23/users/9932ef3f7f874a46b8c07749ce99e095/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:15.941205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:15.960481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957054c-2a63-486a-921f-35adbd99df5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:15.961236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957054c-2a63-486a-921f-35adbd99df5b service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.961614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957054c-2a63-486a-921f-35adbd99df5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.961964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957054c-2a63-486a-921f-35adbd99df5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:15.973111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957054c-2a63-486a-921f-35adbd99df5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:15.975428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957054c-2a63-486a-921f-35adbd99df5b service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:15.993605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1030/2064] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:15.995668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:15.996028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:15.996407 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.000584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:16.017229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee6869-b4d4-4eb0-8560-e61a5fc107c1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.023295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.024703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.025079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.025436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.026220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.028606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] MFA Rules not processed for user `9932ef3f7f874a46b8c07749ce99e095`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:16.033619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.067088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1035/2065] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:16.070765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.089077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7aa9d-a13f-4bdd-83e1-f09c98d5f857 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.095023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.096524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.097064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.097197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.097953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.099944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.119669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1036/2066] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:16.120801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.142789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746841c-4a7a-4a53-bc3e-21e5fa64c708 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.156566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.161962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.162288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.162571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] PATH_INFO: `/v3/projects/fa9da1ba200247caa22e0b1ff84a0da7/users/8d4d8f2c9df14398b2e92334ee55e212/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.197692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1031/2067] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:16.206745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.209901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.231501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9da1ba200247caa22e0b1ff84a0da7, user_id=8d4d8f2c9df14398b2e92334ee55e212, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.239109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.269289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1037/2068] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:16 2022] PUT /identity/v3/projects/fa9da1ba200247caa22e0b1ff84a0da7/users/8d4d8f2c9df14398b2e92334ee55e212/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:16.272951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.340441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d6fee-6f63-4b99-8c87-a14519fd4363 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.376461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73229f80-85d9-48ec-9ea6-d2fa4cf7b67b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.387832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.390208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.390701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.391355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.392104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.393754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.405126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1038/2069] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:16.410235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.424749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd497dea-f7f7-42ac-9a90-9a747a505349 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.429252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74072127-68b1-432a-bfd7-21635f5631fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.430302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74072127-68b1-432a-bfd7-21635f5631fa service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.430692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74072127-68b1-432a-bfd7-21635f5631fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.430985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74072127-68b1-432a-bfd7-21635f5631fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.431170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.432440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.432793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.433143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] PATH_INFO: `/v3/projects/fa9da1ba200247caa22e0b1ff84a0da7/users/8d4d8f2c9df14398b2e92334ee55e212/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.435799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9da1ba200247caa22e0b1ff84a0da7, user_id=8d4d8f2c9df14398b2e92334ee55e212, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.440303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.456477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:16.466469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1039/2070] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:16 2022] PUT /identity/v3/projects/fa9da1ba200247caa22e0b1ff84a0da7/users/8d4d8f2c9df14398b2e92334ee55e212/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:16.469557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.478775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.558953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74072127-68b1-432a-bfd7-21635f5631fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.560509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74072127-68b1-432a-bfd7-21635f5631fa service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.571524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1032/2071] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:16.573025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.573346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.573648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.574556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d7519-b38b-4f9a-8fbd-5764c168712b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.583684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.585634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.585634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.585634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] PATH_INFO: `/v3/projects/9835a5dd19014c4392216eebeed00a86` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.586683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9835a5dd19014c4392216eebeed00a86)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.588774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.623182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] MFA Rules not processed for user `8d4d8f2c9df14398b2e92334ee55e212`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:16.624809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.628041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:16.649329 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9835a5dd19014c4392216eebeed00a86', 'request_id': 'req-a784358f-398b-4854-8a3d-0185ff65c549'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.664808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9835a5dd19014c4392216eebeed00a86', 'request_id': 'req-a784358f-398b-4854-8a3d-0185ff65c549'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.676640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9835a5dd19014c4392216eebeed00a86', 'request_id': 'req-a784358f-398b-4854-8a3d-0185ff65c549'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.693442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] The token cache is being invalidate because project 9835a5dd19014c4392216eebeed00a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:16.693748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.694458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.699079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1040/2072] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:16 2022] DELETE /identity/v3/projects/9835a5dd19014c4392216eebeed00a86 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:16.700398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.702594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.798287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1033/2073] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:16.799494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.801450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.842493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784358f-398b-4854-8a3d-0185ff65c549 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.848308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.849403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.849782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.850123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] PATH_INFO: `/v3/users/181148aa7c0d4fc697e2adfab1fbe887` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.851730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181148aa7c0d4fc697e2adfab1fbe887)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:16.853792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:16.876480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3a59d-8bb1-4697-9a78-a9ccd4fed40c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:16.952039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181148aa7c0d4fc697e2adfab1fbe887'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.952599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:16.953327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:16.953875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:16.954223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:16.957888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:16.961049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181148aa7c0d4fc697e2adfab1fbe887'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.961897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181148aa7c0d4fc697e2adfab1fbe887'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.962726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181148aa7c0d4fc697e2adfab1fbe887'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.968850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:16.977165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181148aa7c0d4fc697e2adfab1fbe887'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:16.990837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1041/2074] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:16 2022] DELETE /identity/v3/users/181148aa7c0d4fc697e2adfab1fbe887 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:16.991660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:16.993018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.037165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:17.053263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.055777 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.062544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a0a75-03d0-43bd-87b0-0ee525e539e0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.066619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.067277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.067558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.067837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.069733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.071116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1034/2075] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:17.142209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.143720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.155553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1042/2076] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:17.307579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.325534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ae9f3-9176-4a4e-96be-acff5dcc1b1d service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.402624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.404366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.427805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.429918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.431285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.431679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] PATH_INFO: `/v3/projects/f35ac0c0323849efb92cc561439823f5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.434049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f35ac0c0323849efb92cc561439823f5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.438321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.500130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d98df2-b254-4ead-9989-7b1315b4dbca service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.506226 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.507787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.507787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.508025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] PATH_INFO: `/v3/users/ad82e8d32cab4dcba32edd95a761a0f2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.509198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad82e8d32cab4dcba32edd95a761a0f2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.512160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.543555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f35ac0c0323849efb92cc561439823f5', 'request_id': 'req-a75b7f21-205c-4721-87af-205505a9cc1d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.557578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f35ac0c0323849efb92cc561439823f5', 'request_id': 'req-a75b7f21-205c-4721-87af-205505a9cc1d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.565113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f35ac0c0323849efb92cc561439823f5', 'request_id': 'req-a75b7f21-205c-4721-87af-205505a9cc1d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.577346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] The token cache is being invalidate because project f35ac0c0323849efb92cc561439823f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:17.577707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.578361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a75b7f21-205c-4721-87af-205505a9cc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.580052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1035/2077] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:17 2022] DELETE /identity/v3/projects/f35ac0c0323849efb92cc561439823f5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:17.581795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad82e8d32cab4dcba32edd95a761a0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.585271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d316-5bfd-4509-83bb-8cff38fd704d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.585720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d316-5bfd-4509-83bb-8cff38fd704d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.586080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8d316-5bfd-4509-83bb-8cff38fd704d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.590094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad82e8d32cab4dcba32edd95a761a0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.590852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad82e8d32cab4dcba32edd95a761a0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.591837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad82e8d32cab4dcba32edd95a761a0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.605973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8d880884-df19-4c15-9b99-c0d9f1dc54c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad82e8d32cab4dcba32edd95a761a0f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:17.606641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-31e8d316-5bfd-4509-83bb-8cff38fd704d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:17.608049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-31e8d316-5bfd-4509-83bb-8cff38fd704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.618569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1043/2078] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:17 2022] DELETE /identity/v3/users/ad82e8d32cab4dcba32edd95a761a0f2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:17.622834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.623116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.623385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.635024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:17.641202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:17.642666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.651430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:17.683701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1036/2079] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:17.693250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.693694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.693838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.708069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1044/2080] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:17.720526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.720966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:17.722127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.722479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.730887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1037/2081] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:17.735064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.744503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e048-3051-490c-8dd8-64a6fe6cd665 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.749535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.750202 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.750483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.750759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.751493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:17.751800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.753399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.769582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1038/2082] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:32:17.772931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.783118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553e53d-4826-4403-8110-59ad34515020 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.784135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141e5b7-3b4b-4359-97f7-61b03236456c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.787872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.788019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.788562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.788742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.788842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.789030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.789127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.789307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.789784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.792000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.799503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.801116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.812974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1045/2083] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:17.824731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1039/2084] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:17 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:17.829216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.841000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85747c5-0475-4a93-a4ab-91cecf283d77 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.913853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.914747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.915792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.916409 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.917104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.917823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.918170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:17.919890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.936924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec6ca5-7bdf-42b7-bdf3-573e41b2a021 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.943034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.943924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.944216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.944482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] PATH_INFO: `/v3/users/3fefa9b5c4064adc90456e5706108343` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.945720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fefa9b5c4064adc90456e5706108343)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.947662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:17.949342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1046/2085] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:17 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:17.953205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:17.963775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4635a-316d-47ad-81e8-0833d1f46da5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:17.980986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:17.982163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:17.982163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:17.982628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:17.990262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:17.992443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.005455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1047/2086] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:18.009745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.014408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fefa9b5c4064adc90456e5706108343'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.022005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6e601-9143-4423-86c1-090dc622c194 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.027536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.028311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.028698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.028985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fefa9b5c4064adc90456e5706108343'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.029112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.029804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fefa9b5c4064adc90456e5706108343'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.030015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.030582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fefa9b5c4064adc90456e5706108343'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.031915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.039063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8818ba13-8e67-487b-9d4d-3647ef18effe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fefa9b5c4064adc90456e5706108343'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.044164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1048/2087] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:18.048332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.050191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.057499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1040/2088] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:17 2022] DELETE /identity/v3/users/3fefa9b5c4064adc90456e5706108343 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:18.062000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7674208-d88a-4940-b3d3-50bc94aec76a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.062326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7674208-d88a-4940-b3d3-50bc94aec76a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.062765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7674208-d88a-4940-b3d3-50bc94aec76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.064109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044642cb-ef28-4384-b2f8-e8947a9e6980 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.077918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:18.086396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d7674208-d88a-4940-b3d3-50bc94aec76a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:18.088762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d7674208-d88a-4940-b3d3-50bc94aec76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.146402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.147319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.147611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.147878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] PATH_INFO: `/v3/projects/686e3458431342f396c8e9394126f355/users/26bce3571c6a43df974b5e6c9bb14f9e/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.170377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=686e3458431342f396c8e9394126f355, user_id=26bce3571c6a43df974b5e6c9bb14f9e, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.173252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.190161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1049/2089] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:18 2022] PUT /identity/v3/projects/686e3458431342f396c8e9394126f355/users/26bce3571c6a43df974b5e6c9bb14f9e/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:18.191989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.201103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.208268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1041/2090] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:18.208268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.208268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.208268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.232072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:18.242908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] MFA Rules not processed for user `26bce3571c6a43df974b5e6c9bb14f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:18.246483 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.285900 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58797-1e1a-4540-a3b4-86840b7e8b4e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.291008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.292181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.292462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.292768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] PATH_INFO: `/v3/projects/e1abb28326074db1a13efed4ec68ee34` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.294130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1abb28326074db1a13efed4ec68ee34)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.296151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.328229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1042/2091] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:18.328229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.328229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.348377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1abb28326074db1a13efed4ec68ee34', 'request_id': 'req-36512476-f7e9-46a4-baff-1f71f240c4f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.356390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1abb28326074db1a13efed4ec68ee34', 'request_id': 'req-36512476-f7e9-46a4-baff-1f71f240c4f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.363140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1abb28326074db1a13efed4ec68ee34', 'request_id': 'req-36512476-f7e9-46a4-baff-1f71f240c4f2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.374707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] The token cache is being invalidate because project e1abb28326074db1a13efed4ec68ee34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:18.375024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.375780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.377339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1050/2092] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:18 2022] DELETE /identity/v3/projects/e1abb28326074db1a13efed4ec68ee34 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:18.378185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.379585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.385353 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b783a-e048-4fd7-889a-ebd88be6cc76 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.413647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:18.436658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.437315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.437630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.437912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.439772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.447519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512476-f7e9-46a4-baff-1f71f240c4f2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.452410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.452555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.453227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.453886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.454170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.454463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.455169 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.457109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.463467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1043/2093] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:18.464237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.465530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.477491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08afd0-a441-44f4-b88f-b517b9bc4451 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.482977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.483796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.484104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.484412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] PATH_INFO: `/v3/users/9932ef3f7f874a46b8c07749ce99e095` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.485628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9932ef3f7f874a46b8c07749ce99e095)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.486349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1051/2094] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:18 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:18.487920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.490861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.550624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9932ef3f7f874a46b8c07749ce99e095'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.557681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:18.563424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1bb32-2d94-4fb5-9618-04fcbc368ab7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.568491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.569337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.569846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.570136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.570918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.573033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.575840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9932ef3f7f874a46b8c07749ce99e095'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.577013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9932ef3f7f874a46b8c07749ce99e095'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.577819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9932ef3f7f874a46b8c07749ce99e095'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.597005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9932ef3f7f874a46b8c07749ce99e095'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:18.606507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1052/2095] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:18 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:18.608573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.612601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.621388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1044/2096] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:18 2022] DELETE /identity/v3/users/9932ef3f7f874a46b8c07749ce99e095 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:18.633163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba5c05-babb-491e-ab20-8e1d611e1f80 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.743530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.744263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.744547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.744905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.746484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.749006 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.764321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1053/2097] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:18.768477 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.784382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f13f1-5372-4897-995b-1dd2c0452c21 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.800806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.802275 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.805376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.805948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.806825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.809838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.833220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1045/2098] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:18.834160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.847190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8079da-451f-4863-a62d-574e0e28794e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:18.852929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6734109-c9d6-49cc-8a4d-12ffa4b8e5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:18.854542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734109-c9d6-49cc-8a4d-12ffa4b8e5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.855117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734109-c9d6-49cc-8a4d-12ffa4b8e5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.855550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734109-c9d6-49cc-8a4d-12ffa4b8e5e0 admin admin] PATH_INFO: `/v3/projects/6034fb2db1744d56905fcc286471002b/users/a288e58516fb46228a8802b2517fc945/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.869268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:18.871358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:18.908755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6734109-c9d6-49cc-8a4d-12ffa4b8e5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6034fb2db1744d56905fcc286471002b, user_id=a288e58516fb46228a8802b2517fc945, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:18.912765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6734109-c9d6-49cc-8a4d-12ffa4b8e5e0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:18.949886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1054/2099] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:18 2022] PUT /identity/v3/projects/6034fb2db1744d56905fcc286471002b/users/a288e58516fb46228a8802b2517fc945/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:18.952319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:18.952613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:18.953025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:18.963687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbadfd-11c5-4330-a782-67e7e9cac93e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.013804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] MFA Rules not processed for user `a288e58516fb46228a8802b2517fc945`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:19.016109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.032798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:19.050973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.052795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.053239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.054172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.057138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.118078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1055/2100] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:19.122696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.159043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.160603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.173609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1046/2101] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:19.174488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.175889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.222607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:19.223906 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4ae5-a2ce-4c73-9c15-e30fe3557df7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.237572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.239607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.239898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.240334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.242089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.245972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.266765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7091b-b80e-457d-943c-ec9503628cf6 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.274738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.275731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.276667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.276667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] PATH_INFO: `/v3/projects/323d66c352254b7a9a033be2c7ea2d23` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.278008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=323d66c352254b7a9a033be2c7ea2d23)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.281345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.295057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1056/2102] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:19 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:19.296190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.379488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '323d66c352254b7a9a033be2c7ea2d23', 'request_id': 'req-ae5283fe-ec39-4d91-bf04-698180b2b889'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.391466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '323d66c352254b7a9a033be2c7ea2d23', 'request_id': 'req-ae5283fe-ec39-4d91-bf04-698180b2b889'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.401238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '323d66c352254b7a9a033be2c7ea2d23', 'request_id': 'req-ae5283fe-ec39-4d91-bf04-698180b2b889'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.417082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] The token cache is being invalidate because project 323d66c352254b7a9a033be2c7ea2d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:19.417639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.418384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ae5283fe-ec39-4d91-bf04-698180b2b889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.424820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1047/2103] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:19 2022] DELETE /identity/v3/projects/323d66c352254b7a9a033be2c7ea2d23 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:19.426649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.426976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.427297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.449801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:19.451216 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.473387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c28e3-7ff0-406e-930a-c6df3103f030 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.560550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1048/2104] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:19.562021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.563905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.573650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:19.580108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a94a23-5649-4293-8e66-df97b9f8c1aa None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.588968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03982255-98db-491a-992c-602772e348c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.591689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03982255-98db-491a-992c-602772e348c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.592947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03982255-98db-491a-992c-602772e348c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.593981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03982255-98db-491a-992c-602772e348c8 admin admin] PATH_INFO: `/v3/projects/7613728bfab941b8ad8a62b74d8edbcd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.595984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03982255-98db-491a-992c-602772e348c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7613728bfab941b8ad8a62b74d8edbcd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.598545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03982255-98db-491a-992c-602772e348c8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.607288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725618a-da83-4b73-95dc-fc9a6759b41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.608847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725618a-da83-4b73-95dc-fc9a6759b41b admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.608847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725618a-da83-4b73-95dc-fc9a6759b41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.609216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725618a-da83-4b73-95dc-fc9a6759b41b admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.610142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725618a-da83-4b73-95dc-fc9a6759b41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.613992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725618a-da83-4b73-95dc-fc9a6759b41b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.652283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1057/2105] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:19 2022] POST /identity/v3/users => generated 298 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:19.653823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.654189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.654467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.666764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7613728bfab941b8ad8a62b74d8edbcd', 'request_id': 'req-03982255-98db-491a-992c-602772e348c8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.681695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7613728bfab941b8ad8a62b74d8edbcd', 'request_id': 'req-03982255-98db-491a-992c-602772e348c8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.682275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:19.683186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:19.685466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.704003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7613728bfab941b8ad8a62b74d8edbcd', 'request_id': 'req-03982255-98db-491a-992c-602772e348c8'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.735697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-03982255-98db-491a-992c-602772e348c8 admin admin] The token cache is being invalidate because project 7613728bfab941b8ad8a62b74d8edbcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:19.735984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.737909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.743565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1049/2106] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:19 2022] DELETE /identity/v3/projects/7613728bfab941b8ad8a62b74d8edbcd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:19.743565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.746713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.789332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1058/2107] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:19.790815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.792983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.809151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915560c3-17b3-4a1b-bba4-1d018a006cc1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.815181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.817009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.817555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.818286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] PATH_INFO: `/v3/users/944f77f3811d4203b62b1d75e64f4a3b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.820211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=944f77f3811d4203b62b1d75e64f4a3b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.822855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.895296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944f77f3811d4203b62b1d75e64f4a3b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.909515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944f77f3811d4203b62b1d75e64f4a3b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.911901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944f77f3811d4203b62b1d75e64f4a3b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.912806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944f77f3811d4203b62b1d75e64f4a3b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.916913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03982255-98db-491a-992c-602772e348c8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:19.931866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:19.932718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:19.934074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:19.934696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:19.937016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:19.940774 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:19.943074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944f77f3811d4203b62b1d75e64f4a3b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:19.965215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1050/2108] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:19.967990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.971386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1059/2109] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:19 2022] DELETE /identity/v3/users/944f77f3811d4203b62b1d75e64f4a3b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:19.974585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:19.976878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:19.979384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:20.017171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae25d-7faf-41a7-9590-6104e309d368 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.111970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:20.139715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19f9c-9e35-40b3-bfd9-44c3476232ba admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.143927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.144752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.145082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.145430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.146172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:20.146480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.148089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.152781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.154655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.154935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.155213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] PATH_INFO: `/v3/projects/a11c74a0d93644cf9f0c9641d00544f9/users/a84dcfa23cda4ed49b6a4ffc4c7879f1/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.164115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1051/2110] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:20.165273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.167803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:20.179532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11c74a0d93644cf9f0c9641d00544f9, user_id=a84dcfa23cda4ed49b6a4ffc4c7879f1, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.182580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.211985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1060/2111] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:19 2022] PUT /identity/v3/projects/a11c74a0d93644cf9f0c9641d00544f9/users/a84dcfa23cda4ed49b6a4ffc4c7879f1/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:20.213306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.230743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853d838-de48-4d12-ac60-73844a633320 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.307053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:20.319927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3dcd-d0f5-456a-90d5-5cd2f16da608 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.325107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.326790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.327213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.328296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.330636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:20.333669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.334377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.334498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.334817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.335435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.337828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.379033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1061/2112] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:20 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:20.379861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.441567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.443053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.454644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1052/2113] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:20.455465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.470516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940a13-02c9-4c41-900f-3fb01cb0ec95 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.491723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb6e7c-f379-41e1-9d83-39197fe9f62a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.498518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.499556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.499685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.500273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.501091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.503072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.515605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1062/2114] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:20.520012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.538800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e046791-6a99-428b-bad9-1b000b3abe82 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.548982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd14098-749c-4e21-9227-ae5946dd4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.551328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd14098-749c-4e21-9227-ae5946dd4acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.551824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd14098-749c-4e21-9227-ae5946dd4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.552602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd14098-749c-4e21-9227-ae5946dd4acb admin admin] PATH_INFO: `/v3/projects/a11c74a0d93644cf9f0c9641d00544f9/users/a84dcfa23cda4ed49b6a4ffc4c7879f1/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.556959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd14098-749c-4e21-9227-ae5946dd4acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11c74a0d93644cf9f0c9641d00544f9, user_id=a84dcfa23cda4ed49b6a4ffc4c7879f1, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.560576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.561461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.561752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.562029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.562730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.564161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd14098-749c-4e21-9227-ae5946dd4acb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.564648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.581069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1063/2115] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:20 2022] PUT /identity/v3/projects/a11c74a0d93644cf9f0c9641d00544f9/users/a84dcfa23cda4ed49b6a4ffc4c7879f1/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:20.595744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1053/2116] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:20 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:20.599981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.600366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.600720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.601281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.620099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1c622-80ee-416a-a883-0618607b00ca admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.642416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] MFA Rules not processed for user `a84dcfa23cda4ed49b6a4ffc4c7879f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:20.643865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:20.672959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:20.726540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.727527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.728036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.728663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.729545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.732742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.752383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1054/2117] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:20.753955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.767172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e21d8d-1600-48ed-a99e-b10ce81c168e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.773828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.774508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.774895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.775845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.776743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1064/2118] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:20.776869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.778572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.780881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.781502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:20.800465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1055/2119] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:20.804440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:20.818304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61063e47-3afa-4117-8034-9b47ed7865cf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.825920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8f758-4c18-4cfd-b02c-d1df631ee9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.827524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f758-4c18-4cfd-b02c-d1df631ee9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.828494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f758-4c18-4cfd-b02c-d1df631ee9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.828824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8f758-4c18-4cfd-b02c-d1df631ee9b1 admin admin] PATH_INFO: `/v3/projects/f76b08583cc74fc398af8bb437bf0c15/users/417871f848f64b0aba95ac4385fe9373/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.855531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8f758-4c18-4cfd-b02c-d1df631ee9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76b08583cc74fc398af8bb437bf0c15, user_id=417871f848f64b0aba95ac4385fe9373, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:20.859519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8f758-4c18-4cfd-b02c-d1df631ee9b1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:20.878200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3735e04-b3ac-40c6-b2ae-666732abc1f7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:20.885507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1056/2120] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:20 2022] PUT /identity/v3/projects/f76b08583cc74fc398af8bb437bf0c15/users/417871f848f64b0aba95ac4385fe9373/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:20.893910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.894228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.894516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.943080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] MFA Rules not processed for user `417871f848f64b0aba95ac4385fe9373`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:20.944312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:20.959110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:20.960617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:20.961007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:20.962459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:20.965680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:21.061270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1057/2121] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:21.062170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.063463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:21.100680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.104148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.117071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1065/2122] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:21.117881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.119380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:21.131423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6964753-e3a6-4436-8b58-706c7c4cfd27 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.135500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b304cb-3c16-4f03-9436-d148faabab5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.136423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b304cb-3c16-4f03-9436-d148faabab5e service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.136772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b304cb-3c16-4f03-9436-d148faabab5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.137116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b304cb-3c16-4f03-9436-d148faabab5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.146435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b304cb-3c16-4f03-9436-d148faabab5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.147966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b304cb-3c16-4f03-9436-d148faabab5e service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.153509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25c184-d66b-4074-aa31-06c55aa2582c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.158962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.159230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1066/2123] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:21.159940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.160239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.160510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] PATH_INFO: `/v3/projects/5a6edce2b99a4b8f929d0cd173328ccc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.161801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6edce2b99a4b8f929d0cd173328ccc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.163786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.248139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6edce2b99a4b8f929d0cd173328ccc', 'request_id': 'req-4619a2c9-8bab-417e-91d6-cad004ca1d83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:21.266948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6edce2b99a4b8f929d0cd173328ccc', 'request_id': 'req-4619a2c9-8bab-417e-91d6-cad004ca1d83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:21.277259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6edce2b99a4b8f929d0cd173328ccc', 'request_id': 'req-4619a2c9-8bab-417e-91d6-cad004ca1d83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:21.282460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:21.302650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] The token cache is being invalidate because project 5a6edce2b99a4b8f929d0cd173328ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:21.303285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:21.304294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4619a2c9-8bab-417e-91d6-cad004ca1d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:21.306607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1058/2124] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:21 2022] DELETE /identity/v3/projects/5a6edce2b99a4b8f929d0cd173328ccc => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:21.312000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.312402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.312816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.355518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:21.361549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:21.402970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:21.507905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1067/2125] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:21.517207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.517736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.518138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.548009 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:21.549379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:21.572525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1059/2126] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:21.574092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.593467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2701c-31c6-438e-910e-5b40b66ea7b8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.602380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.603766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.603766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.603972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.604755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:21.620331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.620331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.652915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1068/2127] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:21.653723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.664110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae8176-7e25-4eca-891e-88aee116850f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.673327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.674121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.674400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.674694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.675395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.685328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.743755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1060/2128] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:21 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:21.747823 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.762592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57628-01c8-47e9-8e13-e45a38702116 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.853213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.854108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.854522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.854792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.856243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.858512 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.893080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1069/2129] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:21 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:21.893632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.911322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e08c6-0525-4c6e-adf5-109dacfccd92 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.924461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.925604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.925937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.926282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.938321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.939977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:21.953750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1061/2130] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:21.957664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:21.971135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605457e7-66cc-4788-a988-f8b05153586d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:21.978450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061f9ca-43a9-4cbf-ba2f-9461a922ab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:21.979347 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f9ca-43a9-4cbf-ba2f-9461a922ab1b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:21.980003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f9ca-43a9-4cbf-ba2f-9461a922ab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:21.980475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f9ca-43a9-4cbf-ba2f-9461a922ab1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:21.981386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061f9ca-43a9-4cbf-ba2f-9461a922ab1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:21.983919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061f9ca-43a9-4cbf-ba2f-9461a922ab1b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:22.003313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1070/2131] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:22.004208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:22.015958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd6b3-01a1-45e3-839e-ddd161785255 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:22.022385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:22.024141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:22.024583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:22.025007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] PATH_INFO: `/v3/projects/241f52355edc44cdae27dbc458388ba3/users/48b04477e359430e87d9615bac81ac45/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:22.046261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=241f52355edc44cdae27dbc458388ba3, user_id=48b04477e359430e87d9615bac81ac45, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:22.049011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:22.076566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1062/2132] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:22 2022] PUT /identity/v3/projects/241f52355edc44cdae27dbc458388ba3/users/48b04477e359430e87d9615bac81ac45/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:22.087405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:22.087726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:22.088125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:22.154982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] MFA Rules not processed for user `48b04477e359430e87d9615bac81ac45`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:22.156477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:22.319262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1071/2133] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:22.334709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:22.336726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:22.426811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e37db8-32b2-422d-b1b7-9ff5286d08dd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:22.430504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1b53a-adbd-43f8-b607-ca489599a547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:22.431223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1b53a-adbd-43f8-b607-ca489599a547 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:22.431565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1b53a-adbd-43f8-b607-ca489599a547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:22.431905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1b53a-adbd-43f8-b607-ca489599a547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:22.441754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1b53a-adbd-43f8-b607-ca489599a547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:22.443276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1b53a-adbd-43f8-b607-ca489599a547 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:22.453209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1063/2134] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:22.474821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:22.496370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:22.559026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345de19-00fb-4e45-b37c-075b8f5eb996 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:22.564171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:22.565132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:22.565434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:22.565832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] PATH_INFO: `/v3/users/48b04477e359430e87d9615bac81ac45` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:22.567149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b04477e359430e87d9615bac81ac45)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:22.571018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:22.637380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b04477e359430e87d9615bac81ac45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:22.654280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b04477e359430e87d9615bac81ac45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:22.654911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b04477e359430e87d9615bac81ac45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:22.655851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b04477e359430e87d9615bac81ac45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:22.678727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b04477e359430e87d9615bac81ac45'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:22.693641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1072/2135] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:22 2022] DELETE /identity/v3/users/48b04477e359430e87d9615bac81ac45 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:22.698029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:22.698322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:22.698604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:22.723383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:22.724627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:22.832598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1064/2136] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:22.878193 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:22.879864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:22.974058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c269d88-a765-4406-a33d-58df11f361dc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:22.974747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:22.975418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:22.976011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:22.976124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:22.995555 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:22.999451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:23.017200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1073/2137] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:23.095223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:23.098398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:23.144116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:23.177626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ce063-f35d-46a5-9834-7c74fa734f11 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:23.181965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:23.182586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.182955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.183213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.185657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:23.265479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:23.267030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f2919-bbbe-4aa8-b641-890ad093a93d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:23.277247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1065/2138] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:23.479185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:23.480688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:23.516261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dfe21-74e9-4861-a152-8bee1552657e service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:23.522600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:23.523514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.523813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.524070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] PATH_INFO: `/v3/projects/241f52355edc44cdae27dbc458388ba3` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.525259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241f52355edc44cdae27dbc458388ba3)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:23.532797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:23.568473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:23.601788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241f52355edc44cdae27dbc458388ba3', 'request_id': 'req-8c455c03-3b97-4b8f-8455-a1b5297da569'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:23.612857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241f52355edc44cdae27dbc458388ba3', 'request_id': 'req-8c455c03-3b97-4b8f-8455-a1b5297da569'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:23.627335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241f52355edc44cdae27dbc458388ba3', 'request_id': 'req-8c455c03-3b97-4b8f-8455-a1b5297da569'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:23.649149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] The token cache is being invalidate because project 241f52355edc44cdae27dbc458388ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:23.649325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:23.650063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8c455c03-3b97-4b8f-8455-a1b5297da569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:23.653493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1074/2139] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:23 2022] DELETE /identity/v3/projects/241f52355edc44cdae27dbc458388ba3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:23.656883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.657355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.659192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.677957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:23.679156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:23.751500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1066/2140] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:23.760215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.760522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.760766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.780011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:23.781225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:23.789350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1075/2141] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:23.793917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:23.804960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ab3f0-d0be-469d-bcb2-0cf3ed945714 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:23.811449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:23.812126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.812449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.812913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.813555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:23.813866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:23.815496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:23.834948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1067/2142] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:23.835890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:23.845678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc6cbf-451c-4a4c-8979-c7d8a30a6cec None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:23.850963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:23.851733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.852202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.852485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.853186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:23.855247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:23.894000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1076/2143] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:23 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:23.898137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:23.917574 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1443d4-f526-4d4d-958f-6c65ee80f179 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:23.988176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:23.989053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:23.989713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:23.990179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:23.992453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:23.994613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:24.018133 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1068/2144] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:23 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:24.029329 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:24.032751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b083f-33b6-4a3a-be76-16a4a08e1cb6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:24.037668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:24.038339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:24.038618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:24.038888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:24.039616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:24.041241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:24.055901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1077/2145] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:24.057060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:24.062308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:24.069100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a41ae-76e1-4528-9a31-e55ed2863b5c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:24.074201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa285e-ea80-4d41-8699-e808ca726283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:24.074863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa285e-ea80-4d41-8699-e808ca726283 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:24.075145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa285e-ea80-4d41-8699-e808ca726283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:24.075474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa285e-ea80-4d41-8699-e808ca726283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:24.076189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa285e-ea80-4d41-8699-e808ca726283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:24.077847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa285e-ea80-4d41-8699-e808ca726283 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:24.087803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1069/2146] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:24.092022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:24.102490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b3224-eafa-4e91-a12c-73d17a94d8e7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:24.108221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:24.109075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:24.109436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:24.111454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] PATH_INFO: `/v3/projects/9c8d2f90128649a38e478dd3f2604894/users/7b32fcb3017c4a4b9c60994efda7cc38/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:24.126240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8d2f90128649a38e478dd3f2604894, user_id=7b32fcb3017c4a4b9c60994efda7cc38, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:24.128825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:24.143849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1078/2147] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:24 2022] PUT /identity/v3/projects/9c8d2f90128649a38e478dd3f2604894/users/7b32fcb3017c4a4b9c60994efda7cc38/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:24.154608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:24.154891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:24.155172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:24.182981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] MFA Rules not processed for user `7b32fcb3017c4a4b9c60994efda7cc38`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:24.184204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:24.261696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1070/2148] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:24.266366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:24.268333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:24.326753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2472ce-2687-4d4e-b643-84368605b4d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:24.330436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:24.331088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:24.331371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:24.331646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:24.340365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:24.342293 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:24.351289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1079/2149] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:25.464553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:25.466047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:25.478123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12d55f-5396-4517-8a85-ed13b45c3b49 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:25.481992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:25.482894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:25.483251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:25.483514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:25.485342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:25.552397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:25.553945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fb505-ec58-46fd-a8b0-61fd4b10bbe4 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:25.564203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1071/2150] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:27.229361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:27.231571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:27.304469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:27.310018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16290eb0-3712-4dc8-aaa8-1c34e71adc00 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:27.315657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:27.316579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:27.317123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:27.317376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] PATH_INFO: `/v3/users/0d001589bb8a41599db42101b09d9c7a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:27.318798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d001589bb8a41599db42101b09d9c7a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:27.320943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:27.378564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d001589bb8a41599db42101b09d9c7a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:27.386708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d001589bb8a41599db42101b09d9c7a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:27.387416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d001589bb8a41599db42101b09d9c7a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:27.388158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d001589bb8a41599db42101b09d9c7a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:27.401762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d001589bb8a41599db42101b09d9c7a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:27.414764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1080/2151] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:27 2022] DELETE /identity/v3/users/0d001589bb8a41599db42101b09d9c7a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:27.419728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:27.420228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:27.420842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:27.439307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:27.440763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:27.521157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1072/2152] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:27.530959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:27.532463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:27.602074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44163b0f-4ba3-4d53-91bd-429d6888fe7f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:27.605779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:27.606434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:27.606748 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:27.607027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:27.616393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:27.617942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:27.628101 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1081/2153] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:27.993036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:27.994604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:28.008617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62fc18-82d0-4ce8-9e25-11e64c230b0b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:28.014423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:28.015299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:28.015604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:28.015850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] PATH_INFO: `/v3/projects/019830178bc945da8d9e6924d1b214a9` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:28.017037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=019830178bc945da8d9e6924d1b214a9)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:28.019171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:28.031461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:28.072534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019830178bc945da8d9e6924d1b214a9', 'request_id': 'req-26adea60-7dd2-4ebe-80be-6084193da1cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.086526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019830178bc945da8d9e6924d1b214a9', 'request_id': 'req-26adea60-7dd2-4ebe-80be-6084193da1cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.089628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019830178bc945da8d9e6924d1b214a9', 'request_id': 'req-26adea60-7dd2-4ebe-80be-6084193da1cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.101176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] The token cache is being invalidate because project 019830178bc945da8d9e6924d1b214a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:28.101500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.102276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.104177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1073/2154] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:27 2022] DELETE /identity/v3/projects/019830178bc945da8d9e6924d1b214a9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:28.108398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:28.119508 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2afe47-2531-4a10-981f-3d1379f1fc46 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:28.203393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:28.204274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:28.204550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:28.204821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] PATH_INFO: `/v3/users/c357ca7929f54efd91082ffeb450d747` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:28.206137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c357ca7929f54efd91082ffeb450d747)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:28.208045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:28.229251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:28.264319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c357ca7929f54efd91082ffeb450d747'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.272585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c357ca7929f54efd91082ffeb450d747'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.273384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c357ca7929f54efd91082ffeb450d747'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.274064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c357ca7929f54efd91082ffeb450d747'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.286388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c357ca7929f54efd91082ffeb450d747'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.299566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1082/2155] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:28 2022] DELETE /identity/v3/users/c357ca7929f54efd91082ffeb450d747 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:28.567261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:28.568865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:28.587475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adea60-7dd2-4ebe-80be-6084193da1cb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:28.658874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:28.660131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:28.660581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:28.661015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] PATH_INFO: `/v3/projects/00a115a9b4144454b4c9be79fabdbf22` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:28.664497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a115a9b4144454b4c9be79fabdbf22)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:28.666960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:28.707668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:28.709814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:28.728828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b2f1-37bb-4d59-9e38-6c8ac7937f52 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:28.729323 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a115a9b4144454b4c9be79fabdbf22', 'request_id': 'req-55fcf8f0-a3cb-4470-8a63-13268b3960f7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.738648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a115a9b4144454b4c9be79fabdbf22', 'request_id': 'req-55fcf8f0-a3cb-4470-8a63-13268b3960f7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.746013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a115a9b4144454b4c9be79fabdbf22', 'request_id': 'req-55fcf8f0-a3cb-4470-8a63-13268b3960f7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.761139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] The token cache is being invalidate because project 00a115a9b4144454b4c9be79fabdbf22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:28.761641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.762627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.764737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1074/2156] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:28 2022] DELETE /identity/v3/projects/00a115a9b4144454b4c9be79fabdbf22 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:28.768535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:28.774271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:28.795779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:28.796661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:28.796943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:28.797219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] PATH_INFO: `/v3/users/52ceb5b9477e431ba4a01adf2f3f485a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:28.798461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ceb5b9477e431ba4a01adf2f3f485a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:28.800389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:28.808398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:28.845918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcf8f0-a3cb-4470-8a63-13268b3960f7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:28.848805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ceb5b9477e431ba4a01adf2f3f485a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.850854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:28.851746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:28.852022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:28.852291 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] PATH_INFO: `/v3/users/8d4d8f2c9df14398b2e92334ee55e212` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:28.853528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4d8f2c9df14398b2e92334ee55e212)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:28.855501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:28.857548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ceb5b9477e431ba4a01adf2f3f485a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.858279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ceb5b9477e431ba4a01adf2f3f485a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.858963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ceb5b9477e431ba4a01adf2f3f485a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.871198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ceb5b9477e431ba4a01adf2f3f485a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.884247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1083/2157] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:28 2022] DELETE /identity/v3/users/52ceb5b9477e431ba4a01adf2f3f485a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:28.904979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4d8f2c9df14398b2e92334ee55e212'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.924203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4d8f2c9df14398b2e92334ee55e212'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.925518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4d8f2c9df14398b2e92334ee55e212'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.926704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4d8f2c9df14398b2e92334ee55e212'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.942166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4d8f2c9df14398b2e92334ee55e212'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:28.960080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1075/2158] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:28 2022] DELETE /identity/v3/users/8d4d8f2c9df14398b2e92334ee55e212 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:29.158309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.164318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:29.204901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:29.271712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f98282-585b-4b6a-8360-5f1dded3c02a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.328394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.329264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.329574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.329839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] PATH_INFO: `/v3/projects/40b09e85301c4e349b6714b13ae2e955` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.331039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b09e85301c4e349b6714b13ae2e955)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.332977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.354673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.357438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:29.376051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ca79d-4c34-4c61-afaf-50ccad14cada admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.385518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b09e85301c4e349b6714b13ae2e955', 'request_id': 'req-ee46a366-a58a-4c07-8971-f08148f91df4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.395315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b09e85301c4e349b6714b13ae2e955', 'request_id': 'req-ee46a366-a58a-4c07-8971-f08148f91df4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.402572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b09e85301c4e349b6714b13ae2e955', 'request_id': 'req-ee46a366-a58a-4c07-8971-f08148f91df4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.416173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] The token cache is being invalidate because project 40b09e85301c4e349b6714b13ae2e955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:29.416460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.417498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ee46a366-a58a-4c07-8971-f08148f91df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.419070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1084/2159] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:29 2022] DELETE /identity/v3/projects/40b09e85301c4e349b6714b13ae2e955 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:29.424124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2a37-47aa-42d1-a40e-e849494b8f7d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.424412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2a37-47aa-42d1-a40e-e849494b8f7d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.424711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e2a37-47aa-42d1-a40e-e849494b8f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.443595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.444463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.444766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.445039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] PATH_INFO: `/v3/projects/fa9da1ba200247caa22e0b1ff84a0da7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.445824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5f9e2a37-47aa-42d1-a40e-e849494b8f7d None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:29.446244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa9da1ba200247caa22e0b1ff84a0da7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.447202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f9e2a37-47aa-42d1-a40e-e849494b8f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:29.448179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.456802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1085/2160] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:29.464524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.464811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.465091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.483797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:29.485017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:29.499864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa9da1ba200247caa22e0b1ff84a0da7', 'request_id': 'req-eae69ab9-9147-45b4-a433-9632e8b3c703'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.501529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:29.509662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa9da1ba200247caa22e0b1ff84a0da7', 'request_id': 'req-eae69ab9-9147-45b4-a433-9632e8b3c703'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.516795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa9da1ba200247caa22e0b1ff84a0da7', 'request_id': 'req-eae69ab9-9147-45b4-a433-9632e8b3c703'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.528083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] The token cache is being invalidate because project fa9da1ba200247caa22e0b1ff84a0da7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:29.528375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.529725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eae69ab9-9147-45b4-a433-9632e8b3c703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:29.530849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:29.531276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1076/2161] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:29 2022] DELETE /identity/v3/projects/fa9da1ba200247caa22e0b1ff84a0da7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:29.536461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5723c1-a575-470f-bf75-9077977058b7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.536753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5723c1-a575-470f-bf75-9077977058b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.537026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5723c1-a575-470f-bf75-9077977058b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.551856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1086/2162] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:29.555842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5c5723c1-a575-470f-bf75-9077977058b7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:29.556010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.557038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5c5723c1-a575-470f-bf75-9077977058b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:29.565108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1077/2163] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:29.565443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b36a5e-43cc-4987-9bb8-6acdad31f5bc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.569923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.570568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.570844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.571115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.571834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:29.572135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.573756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.573866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.574307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.574393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.586782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1087/2164] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:32:29.591203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.592921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:29.594189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:29.601503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598ed41-836e-4674-9a25-e03205dd9fc0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.603531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1078/2165] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:29.607013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.607013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.607749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.607749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.607917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.608063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.609974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.616983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0047d-da0e-4157-91d8-0e5d91cc7623 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.622532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.623756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.624046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.624326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.625066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:29.625382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.627062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.638714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1088/2166] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:29.640689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1079/2167] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:32:29.644880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.645014 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.655744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd2e11-acea-43e8-8f9b-b25c51836d1e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.710863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e5b60-3e1b-4947-a730-33d86a7b6516 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.715296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.715647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.716063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.716358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.716471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.716624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.716918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.717203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.717301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.717931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.719317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.720007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.740249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1080/2168] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:29.744925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.753082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1089/2169] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:29.757226 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.758287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa1d60-4c5d-4aca-8d58-8c09b4c68553 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.826112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca77a41-b63a-48cf-b8fb-e516327d5450 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.830263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.831016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.831139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.831431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.831698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.831832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.832099 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.832502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.832875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.833256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.835061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.835630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.847949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1081/2170] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:29.852139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.859516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1090/2171] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:29.863510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.866983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dc920-2d7b-4b50-afae-512fe74fe172 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.871846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.872697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.873041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.873263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.874020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.875327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3afa4-4b88-41fe-824b-0361513debfd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.875650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.880031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.880766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.881110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.881485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.882397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.884452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.886988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1082/2172] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:29.891254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.901700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1091/2173] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:29.902824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.906886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2127525-69ca-4a24-a99d-85272b3325c3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.913168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a895cca-b9b5-4da8-a73b-c0325020b3cc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.914198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a1b8-f26d-4656-a3bf-3951e450c651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.915204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a1b8-f26d-4656-a3bf-3951e450c651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.915556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a1b8-f26d-4656-a3bf-3951e450c651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.915906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a1b8-f26d-4656-a3bf-3951e450c651 admin admin] PATH_INFO: `/v3/projects/c83afa7a2c074c7c84974fee7a512f0b/users/68c1b4e4bf2a4b84aff4da09bba28f06/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.918661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:29.919637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.919920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.920316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.921401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.924948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.938035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400a1b8-f26d-4656-a3bf-3951e450c651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83afa7a2c074c7c84974fee7a512f0b, user_id=68c1b4e4bf2a4b84aff4da09bba28f06, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:29.941589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1092/2174] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:29.941817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400a1b8-f26d-4656-a3bf-3951e450c651 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:29.943325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:29.951660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:29.957594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005897ab-799a-4df5-aaf3-cd4accdf1b3a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:29.962233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1083/2175] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:29 2022] PUT /identity/v3/projects/c83afa7a2c074c7c84974fee7a512f0b/users/68c1b4e4bf2a4b84aff4da09bba28f06/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:29.968879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741eb8-aff7-473a-9cd4-cb5bd092e14e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:29.969276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741eb8-aff7-473a-9cd4-cb5bd092e14e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:29.969655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741eb8-aff7-473a-9cd4-cb5bd092e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:29.983675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:30.009603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-da741eb8-aff7-473a-9cd4-cb5bd092e14e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:30.012436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-da741eb8-aff7-473a-9cd4-cb5bd092e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:30.140282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:30.141435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:30.142004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:30.142115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] PATH_INFO: `/v3/projects/83eb6b286ea74e0eaaece425cc668ace/users/3503c3ce6d374a20926d7312859eade6/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:30.161206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=83eb6b286ea74e0eaaece425cc668ace, user_id=3503c3ce6d374a20926d7312859eade6, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:30.164505 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:30.176745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1084/2176] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:30.188340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1093/2177] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:29 2022] PUT /identity/v3/projects/83eb6b286ea74e0eaaece425cc668ace/users/3503c3ce6d374a20926d7312859eade6/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:30.203437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:30.203861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:30.204534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:30.232143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:30.234043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:30.278899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] MFA Rules not processed for user `3503c3ce6d374a20926d7312859eade6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:30.281503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:30.317897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c61b-a5ea-4be7-8f93-7bccf00dc8db admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:30.321912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1954c5-9f7c-43a9-95c3-474d7363311d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:30.322637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1954c5-9f7c-43a9-95c3-474d7363311d service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:30.323160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1954c5-9f7c-43a9-95c3-474d7363311d service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:30.323513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1954c5-9f7c-43a9-95c3-474d7363311d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:30.395351 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1085/2178] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:30.399212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1954c5-9f7c-43a9-95c3-474d7363311d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:30.401017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1954c5-9f7c-43a9-95c3-474d7363311d service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:30.409746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:30.411688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:30.413150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1094/2179] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:30.484307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff936ec-3174-446d-843c-6f27ee216ca9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:30.488277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:30.489010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:30.489403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:30.489710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:30.499704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:30.501363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:30.512455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1086/2180] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:32:31.125207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.125527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.125801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.159660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] MFA Rules not processed for user `68c1b4e4bf2a4b84aff4da09bba28f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:31.161177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:31.190340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:31.263972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1095/2181] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:31.268568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:31.283345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddd4e3-efe4-4e75-857d-2dacc5f733cf service glance] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:31.300039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:31.300775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.301063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.301342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.302409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:31.304717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:31.347429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1087/2182] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:31 2022] POST /identity/v3/projects => generated 405 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:31.348612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:31.363029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29152-ef2f-4574-a13a-932d559859e7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:31.486602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:31.488774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.494328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.495873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.502831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:31.506387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:31.550190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1096/2183] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:31 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:31.550481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:31.562406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a1af7-2859-4f56-bea5-1bc14adda1d6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:31.569224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:31.570486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.571138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.571705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.573000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:31.575223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:31.600991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1088/2184] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:31.605681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:31.620531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36e18d-69dd-4381-957f-13223b69f125 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:31.630602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:31.632547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.632958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.633354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] PATH_INFO: `/v3/projects/dba430461c54472d98d284e8152bb85a/users/3765ed2a652b4dd39db1671898f2919a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.659043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba430461c54472d98d284e8152bb85a, user_id=3765ed2a652b4dd39db1671898f2919a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:31.661819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:31.683504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1097/2185] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:31 2022] PUT /identity/v3/projects/dba430461c54472d98d284e8152bb85a/users/3765ed2a652b4dd39db1671898f2919a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:31.688215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:31.694894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:31.702552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9d02d-351b-4ffd-9f8c-127435a88ff4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:31.771737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d907f71-2eb9-4ae5-8c4c-1476c203112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:31.772431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907f71-2eb9-4ae5-8c4c-1476c203112a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.772702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907f71-2eb9-4ae5-8c4c-1476c203112a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.772972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907f71-2eb9-4ae5-8c4c-1476c203112a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.773743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d907f71-2eb9-4ae5-8c4c-1476c203112a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:31.775350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d907f71-2eb9-4ae5-8c4c-1476c203112a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:31.786094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1089/2186] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:31.790384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:31.801008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21970ddd-684c-4c92-aaa6-418b0d0e7b99 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:31.805611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:31.806617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:31.806927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:31.807259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] PATH_INFO: `/v3/projects/dba430461c54472d98d284e8152bb85a/users/3765ed2a652b4dd39db1671898f2919a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:31.809389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba430461c54472d98d284e8152bb85a, user_id=3765ed2a652b4dd39db1671898f2919a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:31.812373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:31.824249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1098/2187] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:31 2022] PUT /identity/v3/projects/dba430461c54472d98d284e8152bb85a/users/3765ed2a652b4dd39db1671898f2919a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:32.477302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:32.477900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:32.477993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:32.511331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] MFA Rules not processed for user `3765ed2a652b4dd39db1671898f2919a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:32.512545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:32.592004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1090/2188] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:33.749913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:33.823920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bb164-b2ec-4f86-88c0-07b83919f286 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:33.829083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:33.830141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:33.830424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:33.830756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] PATH_INFO: `/v3/users/68c1b4e4bf2a4b84aff4da09bba28f06` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:33.832011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c1b4e4bf2a4b84aff4da09bba28f06)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:33.834142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:33.864310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:33.917886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c1b4e4bf2a4b84aff4da09bba28f06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:33.934387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c1b4e4bf2a4b84aff4da09bba28f06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:33.935227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c1b4e4bf2a4b84aff4da09bba28f06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:33.935924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c1b4e4bf2a4b84aff4da09bba28f06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:33.948130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c1b4e4bf2a4b84aff4da09bba28f06'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:33.963034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1099/2189] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:33 2022] DELETE /identity/v3/users/68c1b4e4bf2a4b84aff4da09bba28f06 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:34.234965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:34.255838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99af52f-87de-4f83-9444-8455839c6cfd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:34.317749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:34.318681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:34.318966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:34.319235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] PATH_INFO: `/v3/projects/c83afa7a2c074c7c84974fee7a512f0b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:34.320394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83afa7a2c074c7c84974fee7a512f0b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:34.322347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:34.355515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:34.373271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83afa7a2c074c7c84974fee7a512f0b', 'request_id': 'req-4ef14405-86fc-42d2-a248-18757975f358'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.381922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83afa7a2c074c7c84974fee7a512f0b', 'request_id': 'req-4ef14405-86fc-42d2-a248-18757975f358'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.388301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83afa7a2c074c7c84974fee7a512f0b', 'request_id': 'req-4ef14405-86fc-42d2-a248-18757975f358'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.399082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] The token cache is being invalidate because project c83afa7a2c074c7c84974fee7a512f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:34.399367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.399988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.401525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1091/2190] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:34 2022] DELETE /identity/v3/projects/c83afa7a2c074c7c84974fee7a512f0b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:34.405753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:34.407150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:34.420661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8dd8-3de3-4fcb-80b5-fd8d3b3dff2f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:34.479462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:34.480327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:34.480601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:34.480866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] PATH_INFO: `/v3/users/3765ed2a652b4dd39db1671898f2919a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:34.482247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3765ed2a652b4dd39db1671898f2919a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:34.484181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:34.533879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3765ed2a652b4dd39db1671898f2919a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.543642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3765ed2a652b4dd39db1671898f2919a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.544461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3765ed2a652b4dd39db1671898f2919a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.545096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3765ed2a652b4dd39db1671898f2919a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.557729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-03631c3c-c258-453f-93eb-301154c8c025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3765ed2a652b4dd39db1671898f2919a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.569769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1100/2191] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:34 2022] DELETE /identity/v3/users/3765ed2a652b4dd39db1671898f2919a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:34.750779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:34.752242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:34.765829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef14405-86fc-42d2-a248-18757975f358 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:34.836043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:34.837064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:34.837360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:34.837694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] PATH_INFO: `/v3/projects/dba430461c54472d98d284e8152bb85a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:34.839640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba430461c54472d98d284e8152bb85a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:34.842888 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:34.892967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba430461c54472d98d284e8152bb85a', 'request_id': 'req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.902420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba430461c54472d98d284e8152bb85a', 'request_id': 'req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.909568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba430461c54472d98d284e8152bb85a', 'request_id': 'req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.923237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] The token cache is being invalidate because project dba430461c54472d98d284e8152bb85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:34.923508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.924173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7faffe2f-08a3-43f9-9f5a-3b8ac0ab45b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:34.925809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1092/2192] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:34 2022] DELETE /identity/v3/projects/dba430461c54472d98d284e8152bb85a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:34.930887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:34.931388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:34.931545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:34.951744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:34.952981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:34.967832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:35.025935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1101/2193] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:35.034119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.034412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.034694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.055102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:35.056346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:35.066858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1093/2194] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:35.071957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.083464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c4d47-9339-4893-a4e9-c6001d3c30d5 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.088470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.089122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.089400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.089703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.090428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:35.090733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.092367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.105697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1102/2195] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:32:35.110537 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.121649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12743522-cc30-4d27-a2f2-ea8917e43256 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.127134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.127798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.128078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.128350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.129048 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.131108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.158493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1094/2196] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:35.162661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.173520 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093c700-dfe4-4f64-86b4-204b10f123e0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.244157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.244157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.244157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.244157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.244457 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.246112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.273495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1103/2197] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/users => generated 279 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:35.277972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.290561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080744f-e89d-4281-9ffc-27d28e988a35 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.296498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.297220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.297535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.297809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.298541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.300144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.311545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1095/2198] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:35.315677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.326621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f12e-a5f5-4934-97ae-31a187934229 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.332081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dca6e-3541-48ed-83d4-58ba925be095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.332918 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dca6e-3541-48ed-83d4-58ba925be095 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.333295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dca6e-3541-48ed-83d4-58ba925be095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.333570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dca6e-3541-48ed-83d4-58ba925be095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.334924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dca6e-3541-48ed-83d4-58ba925be095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.337070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dca6e-3541-48ed-83d4-58ba925be095 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.350824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1104/2199] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:35.352255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.360158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:35.364660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e07f5-7902-4512-9139-81cf4da0e2d5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.371432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.373241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.373762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.374183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] PATH_INFO: `/v3/projects/c14109d8b99a4ab1ad0008ad888b3140/users/2c1f5a727a1f4f53b97a5ce38df94fab/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.390772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c14109d8b99a4ab1ad0008ad888b3140, user_id=2c1f5a727a1f4f53b97a5ce38df94fab, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.393915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.412825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1096/2200] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:35 2022] PUT /identity/v3/projects/c14109d8b99a4ab1ad0008ad888b3140/users/2c1f5a727a1f4f53b97a5ce38df94fab/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:35.425893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.426085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.426371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.456211 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] MFA Rules not processed for user `2c1f5a727a1f4f53b97a5ce38df94fab`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:35.457392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:35.535913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1105/2201] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:35.540445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.618571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811d0b-6cf6-4d3f-abc6-3cf5192a6af4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.624143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.624848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.625130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.625459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.626163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.628116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.657195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1097/2202] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:35.661432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.673721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f4ee0-e795-415b-a997-a22e85f063b1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.683040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:35.730579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.732439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.732776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:35.733171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.733494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.733797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.734541 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.736471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.745478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50f5da-cdd2-40b8-ba0f-b43b3d1cb203 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.750685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.751629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.751955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.752234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] PATH_INFO: `/v3/users/417871f848f64b0aba95ac4385fe9373` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.753502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] RBAC: Authorizing `identity:delete_user(user_id=417871f848f64b0aba95ac4385fe9373)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.755445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.757821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1106/2203] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:35.761803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.771772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941a581-4eca-4572-8f37-23c51a08612a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.777116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.777806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.778089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.778673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.779196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.780807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.791503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1107/2204] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:35.796109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.806278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417871f848f64b0aba95ac4385fe9373'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:35.807929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd56cd-f07a-4ab8-bffa-88724f1c925e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.813825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.814924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.815212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.815490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] PATH_INFO: `/v3/projects/9ea73ab493de410aa4ee3a8aa325ea06/users/dcaae4ff29364c8289eca12d0983f5e0/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.820293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417871f848f64b0aba95ac4385fe9373'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:35.821001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417871f848f64b0aba95ac4385fe9373'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:35.821637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417871f848f64b0aba95ac4385fe9373'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:35.830740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e370dcac-8194-426a-8543-ae5f99d1660b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417871f848f64b0aba95ac4385fe9373'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:35.832295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea73ab493de410aa4ee3a8aa325ea06, user_id=dcaae4ff29364c8289eca12d0983f5e0, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.835149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.848192 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1098/2205] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:35 2022] DELETE /identity/v3/users/417871f848f64b0aba95ac4385fe9373 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:35.852532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.852829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.853116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.856453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1108/2206] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:35 2022] PUT /identity/v3/projects/9ea73ab493de410aa4ee3a8aa325ea06/users/dcaae4ff29364c8289eca12d0983f5e0/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:35.861018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.861890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:35.870977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:35.872440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:35.912217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:35.931371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8547a7-abcc-4e9f-94b2-dd04db4ffee4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.936109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.936759 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.937038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.937318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.938116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.939716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.942680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1099/2207] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:35.950452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1109/2208] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:35.952484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.953993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:35.954666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:35.964560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd028-8833-4307-bfd2-f0cdd09e842f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:35.969232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52162dd-9117-4c70-b63c-a3030bdb304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:35.970138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52162dd-9117-4c70-b63c-a3030bdb304e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.970418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52162dd-9117-4c70-b63c-a3030bdb304e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.970692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52162dd-9117-4c70-b63c-a3030bdb304e admin admin] PATH_INFO: `/v3/projects/9ea73ab493de410aa4ee3a8aa325ea06/users/dcaae4ff29364c8289eca12d0983f5e0/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:35.972700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52162dd-9117-4c70-b63c-a3030bdb304e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea73ab493de410aa4ee3a8aa325ea06, user_id=dcaae4ff29364c8289eca12d0983f5e0, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:35.975384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52162dd-9117-4c70-b63c-a3030bdb304e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:35.987714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1110/2209] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:35 2022] PUT /identity/v3/projects/9ea73ab493de410aa4ee3a8aa325ea06/users/dcaae4ff29364c8289eca12d0983f5e0/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:35.998058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:35.998340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:35.998687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.019716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a3838-6812-4d05-ae3a-88b5517517ad None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.032609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] MFA Rules not processed for user `dcaae4ff29364c8289eca12d0983f5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:36.033939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.080364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:36.086360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.087100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.087376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.087650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.117638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1111/2210] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:36.118405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.120040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.155713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.157444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.168356 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1100/2211] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:36.169232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.170643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.197445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd825fce-ce5a-44cf-ac89-589877948f85 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.202917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.203807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.204190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.204477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] PATH_INFO: `/v3/users/26bce3571c6a43df974b5e6c9bb14f9e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.205863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bce3571c6a43df974b5e6c9bb14f9e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.207789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.227783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:36.241837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6023375-91f6-4135-96f1-db2554e15d4c service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.246225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc46c7-054c-4ed1-8f3d-ab9859179459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.248737 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc46c7-054c-4ed1-8f3d-ab9859179459 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.249020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc46c7-054c-4ed1-8f3d-ab9859179459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.249292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc46c7-054c-4ed1-8f3d-ab9859179459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.262527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc46c7-054c-4ed1-8f3d-ab9859179459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.264144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc46c7-054c-4ed1-8f3d-ab9859179459 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.279706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bce3571c6a43df974b5e6c9bb14f9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.284154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1101/2212] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:36.292556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bce3571c6a43df974b5e6c9bb14f9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.293241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bce3571c6a43df974b5e6c9bb14f9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.294320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bce3571c6a43df974b5e6c9bb14f9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.318919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bce3571c6a43df974b5e6c9bb14f9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.372177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1112/2213] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:36 2022] DELETE /identity/v3/users/26bce3571c6a43df974b5e6c9bb14f9e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:36.375067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.375816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.376283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.400329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:36.402586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.430280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.433922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.528189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1102/2214] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:36.533311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc42972-409b-4dcb-ad6d-8291b0332579 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.537384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.538139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.538423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.538734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.540325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.540796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.543076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.554021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:36.630311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.632319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.632560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671008ee-98d0-465b-85cf-ab8d4522dfc3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.637100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c85f0-37d6-4ea5-b325-389b8cded072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.637875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c85f0-37d6-4ea5-b325-389b8cded072 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.638199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c85f0-37d6-4ea5-b325-389b8cded072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.638746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c85f0-37d6-4ea5-b325-389b8cded072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.646057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1113/2215] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:36.652233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c85f0-37d6-4ea5-b325-389b8cded072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.654250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c85f0-37d6-4ea5-b325-389b8cded072 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.668053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1103/2216] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:36.683276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.686558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.707272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa90d02-9b62-4a48-bb6b-c5a1c8ead6b3 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.712469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.713355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.713700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.713981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] PATH_INFO: `/v3/projects/f76b08583cc74fc398af8bb437bf0c15` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.715184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76b08583cc74fc398af8bb437bf0c15)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.718079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.782536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76b08583cc74fc398af8bb437bf0c15', 'request_id': 'req-ba67cdbc-83d5-4277-8311-7333fad70f02'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.793905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76b08583cc74fc398af8bb437bf0c15', 'request_id': 'req-ba67cdbc-83d5-4277-8311-7333fad70f02'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.803972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76b08583cc74fc398af8bb437bf0c15', 'request_id': 'req-ba67cdbc-83d5-4277-8311-7333fad70f02'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.818462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] The token cache is being invalidate because project f76b08583cc74fc398af8bb437bf0c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:36.818769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.819631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ba67cdbc-83d5-4277-8311-7333fad70f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:36.824984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1114/2217] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:36 2022] DELETE /identity/v3/projects/f76b08583cc74fc398af8bb437bf0c15 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:36.826681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.826999 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.827276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.853977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:36.855854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.927823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1104/2218] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:36.935808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d922-c960-4613-b862-e54e339c10db None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.936313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d922-c960-4613-b862-e54e339c10db None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.936593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820d922-c960-4613-b862-e54e339c10db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.956387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8820d922-c960-4613-b862-e54e339c10db None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:36.957803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8820d922-c960-4613-b862-e54e339c10db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:36.966603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1115/2219] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:36.971610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.983219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f0bc9-af3c-4f8f-9c65-085c42eecc37 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:36.989257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:36.989978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:36.990262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:36.990554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:36.991290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:36.991640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:36.993237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:36.996891 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820d922-c960-4613-b862-e54e339c10db None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:36.998980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8820d922-c960-4613-b862-e54e339c10db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.005793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1105/2220] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:32:37.009964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.016476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820d922-c960-4613-b862-e54e339c10db None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.020813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f7d9a-4c26-4a8e-9ea9-c73d2e11175d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.022613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.023671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.024037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.024361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] PATH_INFO: `/v3/projects/686e3458431342f396c8e9394126f355` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.025621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=686e3458431342f396c8e9394126f355)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.026132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.027042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.027420 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.027608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.027877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.028749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.030848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.050294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:37.068447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1106/2221] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:37 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:37.072498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.087777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686e3458431342f396c8e9394126f355', 'request_id': 'req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.095810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686e3458431342f396c8e9394126f355', 'request_id': 'req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.104926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686e3458431342f396c8e9394126f355', 'request_id': 'req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.117281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] The token cache is being invalidate because project 686e3458431342f396c8e9394126f355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:37.117691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.118425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.120066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1116/2222] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:36 2022] DELETE /identity/v3/projects/686e3458431342f396c8e9394126f355 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:37.124103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.125563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.140035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:37.141646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ca5ee-1ede-4e8d-890e-eca6a4dc3923 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.201662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdab3b-8d09-47e0-bab1-8c8dc1b13e71 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.206738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.207124 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.207750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.207914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.208070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.208223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.208352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] PATH_INFO: `/v3/users/a288e58516fb46228a8802b2517fc945` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.208511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.209255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.209942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a288e58516fb46228a8802b2517fc945)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.211343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.212146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.237742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1107/2223] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:37 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:37.239364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.242652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.256815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688023d7-f2e4-4cff-a26e-cc7dfbd5f30b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.261807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.262455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.263084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.263713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.264468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.266116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.276126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a288e58516fb46228a8802b2517fc945'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.276388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1108/2224] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:37.280634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.286170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a288e58516fb46228a8802b2517fc945'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.286977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a288e58516fb46228a8802b2517fc945'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.287610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a288e58516fb46228a8802b2517fc945'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.291572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715774d-2c40-4a52-8862-950239858ffd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.297405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.298090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.298376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.298654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.299396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.300997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.301104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-c2193502-c832-4853-bfae-6ac0834435be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a288e58516fb46228a8802b2517fc945'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.312551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1109/2225] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:37.314009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1117/2226] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:37 2022] DELETE /identity/v3/users/a288e58516fb46228a8802b2517fc945 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:37.322256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.324146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.337991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53db5-c6e5-44ad-8d51-d34db55a0926 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.410185 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.411182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.411683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.412024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] PATH_INFO: `/v3/projects/5a26987661f54291b3cf7086b0a1c98c/users/d9a8a480afb3438cb269ebd710c78018/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.430667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a26987661f54291b3cf7086b0a1c98c, user_id=d9a8a480afb3438cb269ebd710c78018, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.434085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.437562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:37.454646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1110/2227] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:37 2022] PUT /identity/v3/projects/5a26987661f54291b3cf7086b0a1c98c/users/d9a8a480afb3438cb269ebd710c78018/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:37.459796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.460017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.460327 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.485139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:37.486561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.549073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1118/2228] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:37.558975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.561032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.619815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.621256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:37.637919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c288a-a7a9-4849-9d56-2a1937473359 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.644444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.645469 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.645600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.645894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] PATH_INFO: `/v3/projects/6034fb2db1744d56905fcc286471002b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.647154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6034fb2db1744d56905fcc286471002b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.649159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.674793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c2b4d-6d20-4089-83c7-a260f21e66f9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.685127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.686008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.686438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.686879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.695855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:37.701051 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.703252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:37.719217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1111/2229] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:37.730925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6034fb2db1744d56905fcc286471002b', 'request_id': 'req-236f4e69-b763-427b-abf1-0a1ac21dbeb5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.739554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6034fb2db1744d56905fcc286471002b', 'request_id': 'req-236f4e69-b763-427b-abf1-0a1ac21dbeb5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.747611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6034fb2db1744d56905fcc286471002b', 'request_id': 'req-236f4e69-b763-427b-abf1-0a1ac21dbeb5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.763642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] The token cache is being invalidate because project 6034fb2db1744d56905fcc286471002b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:37.763944 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.764782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:37.766673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1119/2230] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:37 2022] DELETE /identity/v3/projects/6034fb2db1744d56905fcc286471002b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:37.771031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:37.782984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88616-c839-40e7-8ed5-a4a6491b5d03 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:37.874408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:37.875386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:37.875799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:37.876082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] PATH_INFO: `/v3/users/a84dcfa23cda4ed49b6a4ffc4c7879f1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:37.877403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84dcfa23cda4ed49b6a4ffc4c7879f1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:37.879551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:38.018315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84dcfa23cda4ed49b6a4ffc4c7879f1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.054626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84dcfa23cda4ed49b6a4ffc4c7879f1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.055597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84dcfa23cda4ed49b6a4ffc4c7879f1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.056668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84dcfa23cda4ed49b6a4ffc4c7879f1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.082262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5276f800-a8d9-43f0-ad16-6fa62bf48f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84dcfa23cda4ed49b6a4ffc4c7879f1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.123095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1112/2231] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:37 2022] DELETE /identity/v3/users/a84dcfa23cda4ed49b6a4ffc4c7879f1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:38.540282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:38.542532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:38.556992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f4e69-b763-427b-abf1-0a1ac21dbeb5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:38.618436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:38.619462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:38.619757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:38.620063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] PATH_INFO: `/v3/projects/a11c74a0d93644cf9f0c9641d00544f9` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:38.621258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11c74a0d93644cf9f0c9641d00544f9)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:38.623750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:38.674153 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11c74a0d93644cf9f0c9641d00544f9', 'request_id': 'req-3705fe6f-7556-4e01-ba75-e81312cebab6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.682140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11c74a0d93644cf9f0c9641d00544f9', 'request_id': 'req-3705fe6f-7556-4e01-ba75-e81312cebab6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.689567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11c74a0d93644cf9f0c9641d00544f9', 'request_id': 'req-3705fe6f-7556-4e01-ba75-e81312cebab6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.701517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] The token cache is being invalidate because project a11c74a0d93644cf9f0c9641d00544f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:38.702106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.703143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-3705fe6f-7556-4e01-ba75-e81312cebab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:38.706455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1120/2232] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:38 2022] DELETE /identity/v3/projects/a11c74a0d93644cf9f0c9641d00544f9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:38.711680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04e40-7d16-4df6-a869-de5e1e910ba6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:38.712421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04e40-7d16-4df6-a869-de5e1e910ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:38.712992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04e40-7d16-4df6-a869-de5e1e910ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:38.744869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-53c04e40-7d16-4df6-a869-de5e1e910ba6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:38.746959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-53c04e40-7d16-4df6-a869-de5e1e910ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:38.767708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:38.771318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4988-9b45-410c-8aca-e943a219722d None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:38.771780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4988-9b45-410c-8aca-e943a219722d None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:38.772467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4988-9b45-410c-8aca-e943a219722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:38.806779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-427a4988-9b45-410c-8aca-e943a219722d None None] MFA Rules not processed for user `d9a8a480afb3438cb269ebd710c78018`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:38.807976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-427a4988-9b45-410c-8aca-e943a219722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:38.842849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1113/2233] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:38.851205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:38.851399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:38.851815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:38.852158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:38.883518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:38.884767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:38.893669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1114/2234] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:38.897587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:38.898762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1121/2235] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:38.908049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a90ea3-75aa-4093-b203-92516d0f1039 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:38.910498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4988-9b45-410c-8aca-e943a219722d None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:38.912002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-427a4988-9b45-410c-8aca-e943a219722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:38.913476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:38.914198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:38.914554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:38.914914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:38.915873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:38.916502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:38.919136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:38.932026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1115/2236] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:38.936435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:38.950846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf0846-cad9-4170-a755-1ca7c63c4578 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:38.960642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:38.961928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:38.962872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:38.963874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:38.964884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:38.969147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:38.995514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4988-9b45-410c-8aca-e943a219722d None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:38.999291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:38.999948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.000229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.000493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.007764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1116/2237] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:38 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:32:39.008985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.011859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.013574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.022833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1122/2238] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:39.101958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d762a-4de9-4a25-b057-9acc50b0d9bd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:39.110126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:39.110816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.111103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.111383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.112087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.114075 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.141696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1117/2239] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:39 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:39.142984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.153924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c83a6-0283-4bb3-adc7-d8816bf6f561 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:39.159311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:39.160026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.160291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.160601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.161374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.163048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.174287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1123/2240] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:39.179359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.194167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582a62e-dde8-4653-becd-6bd57a8e6f01 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:39.202145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0142c-3df0-498c-9590-86524c65ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:39.202145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0142c-3df0-498c-9590-86524c65ed36 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.202145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0142c-3df0-498c-9590-86524c65ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.202145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0142c-3df0-498c-9590-86524c65ed36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.202793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0142c-3df0-498c-9590-86524c65ed36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.204990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0142c-3df0-498c-9590-86524c65ed36 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.216436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1118/2241] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:39.220697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.232226 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9280796-5d4c-4db0-ad89-9b3e6238fa96 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:39.243862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:39.244778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.245062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.245339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] PATH_INFO: `/v3/projects/5cad512cfbe34d71a7b5091b6aa93c54/users/9b41bb6f796d4f77b6aedbd8fc873ff6/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.263682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cad512cfbe34d71a7b5091b6aa93c54, user_id=9b41bb6f796d4f77b6aedbd8fc873ff6, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.266535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.283635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1124/2242] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:39 2022] PUT /identity/v3/projects/5cad512cfbe34d71a7b5091b6aa93c54/users/9b41bb6f796d4f77b6aedbd8fc873ff6/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:39.294804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.295148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.295358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.329994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] MFA Rules not processed for user `9b41bb6f796d4f77b6aedbd8fc873ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:39.331682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:39.398110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:39.427782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1119/2243] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:39.447126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.506745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42a36-1a41-4f6d-817c-0b215f7e7f53 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:39.510821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023062-ad51-4a8b-87bd-5a35b07d34c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:39.511506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023062-ad51-4a8b-87bd-5a35b07d34c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.511785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023062-ad51-4a8b-87bd-5a35b07d34c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.512056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023062-ad51-4a8b-87bd-5a35b07d34c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.526998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed023062-ad51-4a8b-87bd-5a35b07d34c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.528500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed023062-ad51-4a8b-87bd-5a35b07d34c6 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.541174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1125/2244] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:39.598072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.735385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c31c4-6f5b-41dd-a8c7-0ee398d6f2e9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:39.740354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:39.741549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.741781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.742283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] PATH_INFO: `/v3/users/9b41bb6f796d4f77b6aedbd8fc873ff6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.744077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b41bb6f796d4f77b6aedbd8fc873ff6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:39.746063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:39.805920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b41bb6f796d4f77b6aedbd8fc873ff6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:39.833650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b41bb6f796d4f77b6aedbd8fc873ff6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:39.834620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b41bb6f796d4f77b6aedbd8fc873ff6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:39.835369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b41bb6f796d4f77b6aedbd8fc873ff6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:39.844624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b41bb6f796d4f77b6aedbd8fc873ff6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:39.858485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1120/2245] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:39 2022] DELETE /identity/v3/users/9b41bb6f796d4f77b6aedbd8fc873ff6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:39.863174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:39.863493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:39.863916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:39.881704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:39.882963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:39.923733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:39.950585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1126/2246] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:39.962637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:39.964204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:40.023091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6c329-1f99-41e4-99df-bd8f8a7188c6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.026818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ede4a-3952-4785-9cec-9946874c704f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.027457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ede4a-3952-4785-9cec-9946874c704f service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.027736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ede4a-3952-4785-9cec-9946874c704f service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.028008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ede4a-3952-4785-9cec-9946874c704f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.038140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ede4a-3952-4785-9cec-9946874c704f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.039630 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ede4a-3952-4785-9cec-9946874c704f service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.049951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1121/2247] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:40.336310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:40.338126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:40.357001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8f2c5-6ae7-42ef-ad80-caa0ab84b870 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.361869 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.362826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.363196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.363317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] PATH_INFO: `/v3/projects/5cad512cfbe34d71a7b5091b6aa93c54` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.364475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cad512cfbe34d71a7b5091b6aa93c54)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.366545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.423938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cad512cfbe34d71a7b5091b6aa93c54', 'request_id': 'req-309bb84c-263d-427d-bd83-bb24dc64aa13'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:40.435602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cad512cfbe34d71a7b5091b6aa93c54', 'request_id': 'req-309bb84c-263d-427d-bd83-bb24dc64aa13'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:40.444669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cad512cfbe34d71a7b5091b6aa93c54', 'request_id': 'req-309bb84c-263d-427d-bd83-bb24dc64aa13'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:40.447848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cd48b-78e8-4ab5-9b56-1d7f39444300 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.447848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cd48b-78e8-4ab5-9b56-1d7f39444300 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.447848 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277cd48b-78e8-4ab5-9b56-1d7f39444300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.454339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:40.459734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] The token cache is being invalidate because project 5cad512cfbe34d71a7b5091b6aa93c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:40.460081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:40.461518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-309bb84c-263d-427d-bd83-bb24dc64aa13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:40.464206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1127/2248] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:40 2022] DELETE /identity/v3/projects/5cad512cfbe34d71a7b5091b6aa93c54 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:40.473456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.473735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.474012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.474110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-277cd48b-78e8-4ab5-9b56-1d7f39444300 None None] MFA Rules not processed for user `d9a8a480afb3438cb269ebd710c78018`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:40.475258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-277cd48b-78e8-4ab5-9b56-1d7f39444300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:40.493980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:40.495208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:40.532712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1122/2249] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:40.565556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1128/2250] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:40.571472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.571759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.572032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.591087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:40.592457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:40.600452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1123/2251] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:40.604849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:40.615044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44483212-ce37-4417-8cc9-7975e3e5b1cc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.620199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.620932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.621220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.621542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.625993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:40.626337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.628025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.641342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1129/2252] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:40.645465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:40.655985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184a0f3-b019-4e7f-86b7-be5f4e2c868f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.665743 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c857f-5c70-4a9b-a189-3975b8386ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.666939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c857f-5c70-4a9b-a189-3975b8386ded admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.667831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c857f-5c70-4a9b-a189-3975b8386ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.668118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c857f-5c70-4a9b-a189-3975b8386ded admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.668885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c857f-5c70-4a9b-a189-3975b8386ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.671000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c857f-5c70-4a9b-a189-3975b8386ded admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.711665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1124/2253] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:40 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:40.721758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:40.732880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609a279-e2eb-411c-a514-820ad94ea043 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.813235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:40.821778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec809e-7d00-44c0-9d54-088fd2bc5040 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.825228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.826033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.826600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.827017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.827401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec809e-7d00-44c0-9d54-088fd2bc5040 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.827741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.829997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec809e-7d00-44c0-9d54-088fd2bc5040 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.830361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.832305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1125/2254] 10.222.0.24 () {54 vars in 912 bytes} [Thu May 19 13:32:40 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:32:40.846895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.847559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.847972 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.874941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1130/2255] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:40 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:40.878317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:40.894192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ec7dc-a48a-414b-b765-a266c4e03bf7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.900280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.901628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.902868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.912544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.914255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.916933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.918096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:40.919862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:40.949005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1131/2256] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:40.959989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:40.973378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88a144-504b-4ad8-af76-e9786ff42017 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:40.980546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:40.981467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:40.981653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:40.982304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:40.983411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:40.985047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:40.997741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1132/2257] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:41.002564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:41.028330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7e4d3-31b8-4045-9c9f-2004cac34125 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:41.034282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44d013-158c-40ea-b099-8353f2912e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:41.035283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44d013-158c-40ea-b099-8353f2912e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:41.035634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44d013-158c-40ea-b099-8353f2912e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:41.035980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44d013-158c-40ea-b099-8353f2912e60 admin admin] PATH_INFO: `/v3/projects/510387333865470196f9be30ef62395c/users/b11df35c0e4541deaea0494759301a94/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:41.040787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1126/2258] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:41.056172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:41.057815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44d013-158c-40ea-b099-8353f2912e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510387333865470196f9be30ef62395c, user_id=b11df35c0e4541deaea0494759301a94, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:41.058293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:41.061138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44d013-158c-40ea-b099-8353f2912e60 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:41.072054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc806de-aba2-48a8-bc73-a39ee23142cf None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:41.084751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1133/2259] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:41 2022] PUT /identity/v3/projects/510387333865470196f9be30ef62395c/users/b11df35c0e4541deaea0494759301a94/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:41.090794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5556d5-8846-410e-bb4b-c2d54e70c0d2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:41.090794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5556d5-8846-410e-bb4b-c2d54e70c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:41.091536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5556d5-8846-410e-bb4b-c2d54e70c0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:41.128345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ed5556d5-8846-410e-bb4b-c2d54e70c0d2 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:41.129776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ed5556d5-8846-410e-bb4b-c2d54e70c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:41.139734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:41.140790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:41.141225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:41.141750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:41.157000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:41.158752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:41.181272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1127/2260] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:41.220208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1134/2261] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:41.229736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:41.231723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:41.251174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89740c3c-c36a-4347-aee3-cb7a51578e5b service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:41.258262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:41.258262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:41.258262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:41.258262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:41.269316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:41.271118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:41.282948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1128/2262] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:45.934124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9f6e3-06c0-404b-9fca-95e8dab7bdbc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:45.934713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9f6e3-06c0-404b-9fca-95e8dab7bdbc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:45.934791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9f6e3-06c0-404b-9fca-95e8dab7bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:45.955408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-70c9f6e3-06c0-404b-9fca-95e8dab7bdbc None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:45.956628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70c9f6e3-06c0-404b-9fca-95e8dab7bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:46.024471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1135/2263] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:32:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:46.114239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:46.115830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:46.182511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:46.287637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565c83b-2faa-4e1a-8508-c30e100a5d8c service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:46.292592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:46.293390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:46.293704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:46.294071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:46.304437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:46.306507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:46.316719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1129/2264] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:32:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:32:47.289901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2eb44-5b2f-4b9f-92ee-e523ef3403c6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:47.290443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2eb44-5b2f-4b9f-92ee-e523ef3403c6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:47.290443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2eb44-5b2f-4b9f-92ee-e523ef3403c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:47.314137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6fe2eb44-5b2f-4b9f-92ee-e523ef3403c6 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:47.315856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6fe2eb44-5b2f-4b9f-92ee-e523ef3403c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:47.328887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1136/2265] 10.222.0.24 () {58 vars in 967 bytes} [Thu May 19 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:47.342766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:47.354871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f335b-b414-403d-bb16-84917303d751 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:47.361494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:47.362586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:47.363027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:47.363455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:47.375641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:47.377678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:47.391045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1130/2266] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:32:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:32:47.714368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632c7a7-e289-4f9d-8d2e-07dbc75760d1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:47.714886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632c7a7-e289-4f9d-8d2e-07dbc75760d1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:47.715358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632c7a7-e289-4f9d-8d2e-07dbc75760d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:47.738313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b632c7a7-e289-4f9d-8d2e-07dbc75760d1 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:47.760489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b632c7a7-e289-4f9d-8d2e-07dbc75760d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:47.798289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1137/2267] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:47.819668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:47.850920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8262c-cabf-4339-9217-2c511ae456ad service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:47.873735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:47.875327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:47.875610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:47.876162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:47.892936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:47.895155 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:47.908357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1131/2268] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:32:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:32:48.287158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:48.287359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:48.287648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:48.346073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] MFA Rules not processed for user `b11df35c0e4541deaea0494759301a94`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:48.348389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:48.416225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:48.515784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1138/2269] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:48.515879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:48.534034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62cb57-0d46-4a08-ac24-2dd451fe2e44 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:48.545240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:48.545240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:48.558495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:48.559168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:48.559998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:48.560274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:48.562466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:48.569017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:48.572550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158730a2-c7de-473c-ad69-a60a79eba554 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:48.587875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:48.588784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:48.590047 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:48.591230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] PATH_INFO: `/v3/users/7b32fcb3017c4a4b9c60994efda7cc38` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:48.593809 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b32fcb3017c4a4b9c60994efda7cc38)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:48.598972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:48.639089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1132/2270] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:48 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:48.639993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:48.695107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b32fcb3017c4a4b9c60994efda7cc38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:48.711234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b32fcb3017c4a4b9c60994efda7cc38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:48.712399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b32fcb3017c4a4b9c60994efda7cc38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:48.713404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b32fcb3017c4a4b9c60994efda7cc38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:48.735185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f72ec1d6-cfeb-4ec5-96ef-327075f29fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b32fcb3017c4a4b9c60994efda7cc38'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:48.761001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab421d0-a3de-45cb-b4b5-ea6365af202e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:48.766448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1139/2271] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:48 2022] DELETE /identity/v3/users/7b32fcb3017c4a4b9c60994efda7cc38 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:48.769754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:48.769991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8b488-65c0-44a1-80c6-ad530f6f6cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:48.770199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:48.770690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b488-65c0-44a1-80c6-ad530f6f6cea admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:48.770818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:48.770974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b488-65c0-44a1-80c6-ad530f6f6cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:48.771371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8b488-65c0-44a1-80c6-ad530f6f6cea admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:48.772078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8b488-65c0-44a1-80c6-ad530f6f6cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:48.774198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8b488-65c0-44a1-80c6-ad530f6f6cea admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:48.791506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:48.793266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:48.805058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1133/2272] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:48 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:48.806831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:48.807272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:48.807981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:48.832045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:48.833212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:48.888468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1140/2273] 10.222.0.24 () {58 vars in 967 bytes} [Thu May 19 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:48.889548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:48.891476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:48.895726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:48.924805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1134/2274] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:48.925910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:48.928044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:48.967342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:49.010647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b318c89-3b36-495e-b598-95755070bd78 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:49.020397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:49.021492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:49.021857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:49.022208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:49.035719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:49.037917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:49.044786 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6e012-a562-4389-be8c-5bdc581f2a01 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:49.057828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:49.062331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:49.062847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:49.063260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:49.064084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1135/2275] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:32:49.065134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:49.074276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:49.086709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:49.090920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:49.123359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1141/2276] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:49.126836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:49.152313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463a24-c18b-43f9-bf86-c386d4a17443 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:49.160536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:49.161161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:49.161738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:49.162127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] PATH_INFO: `/v3/projects/aa772f930d6a41bbab9ccfc2bd4d8fd0/users/9236c3e6183145d0a873b7c59754087b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:49.203948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38a22f-ff92-42a4-9dc6-ff8dfdbbaee8 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:49.205647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa772f930d6a41bbab9ccfc2bd4d8fd0, user_id=9236c3e6183145d0a873b7c59754087b, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:49.209603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:49.231013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:49.231725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:49.232212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:49.232497 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:49.243516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:49.247798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:49.249214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1142/2277] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:49 2022] PUT /identity/v3/projects/aa772f930d6a41bbab9ccfc2bd4d8fd0/users/9236c3e6183145d0a873b7c59754087b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:49.253172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:49.284430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1136/2278] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:49.481846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b95d7-7fd4-40a8-9adc-fb1c359d8f34 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:49.497146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:49.498650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:49.499260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:49.501172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:49.502949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:49.506466 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:49.538968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:49.539465 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1143/2279] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:32:49.552104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b8429-c2af-45fb-9408-63a1a4437bf9 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:49.564226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9c504-99a4-49cd-b5f7-e024a8958b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:49.565938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9c504-99a4-49cd-b5f7-e024a8958b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:49.566636 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9c504-99a4-49cd-b5f7-e024a8958b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:49.568442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9c504-99a4-49cd-b5f7-e024a8958b8b admin admin] PATH_INFO: `/v3/projects/aa772f930d6a41bbab9ccfc2bd4d8fd0/users/9236c3e6183145d0a873b7c59754087b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:49.572362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9c504-99a4-49cd-b5f7-e024a8958b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa772f930d6a41bbab9ccfc2bd4d8fd0, user_id=9236c3e6183145d0a873b7c59754087b, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:49.575721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9c504-99a4-49cd-b5f7-e024a8958b8b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:49.617875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1137/2280] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:49 2022] PUT /identity/v3/projects/aa772f930d6a41bbab9ccfc2bd4d8fd0/users/9236c3e6183145d0a873b7c59754087b/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:49.995827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:49.998574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:50.041447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f5d79-37a4-4f54-878d-a8c09c6315d0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:50.198158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:50.199495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:50.199877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:50.200348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] PATH_INFO: `/v3/projects/9c8d2f90128649a38e478dd3f2604894` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:50.202519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8d2f90128649a38e478dd3f2604894)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:50.212213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:50.333657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:50.337956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8d2f90128649a38e478dd3f2604894', 'request_id': 'req-7ac42489-70ef-4ca6-a485-2bb59d230da9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:50.377958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8d2f90128649a38e478dd3f2604894', 'request_id': 'req-7ac42489-70ef-4ca6-a485-2bb59d230da9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:50.386064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8d2f90128649a38e478dd3f2604894', 'request_id': 'req-7ac42489-70ef-4ca6-a485-2bb59d230da9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:50.403530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] The token cache is being invalidate because project 9c8d2f90128649a38e478dd3f2604894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:50.405629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:50.409837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7ac42489-70ef-4ca6-a485-2bb59d230da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:50.413535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1144/2281] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:49 2022] DELETE /identity/v3/projects/9c8d2f90128649a38e478dd3f2604894 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:50.418531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:50.419062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:50.419274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:50.443873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:50.445157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:50.593439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1138/2282] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:50.603078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:50.603727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:50.603843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:50.633773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:50.639325 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:50.660889 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1145/2283] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:50.665250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:50.690501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c9dfc-cd9d-4d55-aec9-d6458a8aed0a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:50.704243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:50.705273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:50.705702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:50.706237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:50.707151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:32:50.707566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:50.709535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:50.734601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:50.757353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:50.773342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1139/2284] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:32:50.789322 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9740ef-5559-420c-82e2-814e882adfaf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:50.798507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:50.798507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:50.798870 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:50.799271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:50.800072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:50.803146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:50.843988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1146/2285] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:32:50 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:50.847845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:50.862509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2496226-e028-402a-973c-033bc3d72199 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:51.017194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:51.018017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:51.018606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:51.018963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:51.020395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:51.026326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:51.076490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1140/2286] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:32:50 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:32:51.076858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:51.088983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3560a4-b711-411e-b7b0-9096c9848dcf admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:51.098680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:51.098680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:51.098680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:51.098680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:51.099029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:51.099704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:51.121395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1147/2287] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:51.125981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:51.153720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de6e58-c2a0-4134-8623-02199ce4b5fb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:51.173076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdc5e4-e193-4709-bc58-b968b833169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:51.173815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdc5e4-e193-4709-bc58-b968b833169b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:51.174368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdc5e4-e193-4709-bc58-b968b833169b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:51.174657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdc5e4-e193-4709-bc58-b968b833169b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:51.176217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdc5e4-e193-4709-bc58-b968b833169b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:51.178448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfdc5e4-e193-4709-bc58-b968b833169b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:51.206498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1141/2288] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:32:51.210728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:51.223872 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35473a7-8d9c-4b30-82bd-b0e87b932019 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:51.234055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:51.235727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:51.236468 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:51.238032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] PATH_INFO: `/v3/projects/47bad3b31c7f48578466b592680befaf/users/28be028abef24acaa41795a50c267ff3/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:51.272656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bad3b31c7f48578466b592680befaf, user_id=28be028abef24acaa41795a50c267ff3, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:51.280281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:51.311075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1148/2289] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:32:51 2022] PUT /identity/v3/projects/47bad3b31c7f48578466b592680befaf/users/28be028abef24acaa41795a50c267ff3/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:32:51.321615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:51.322449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:51.322974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:51.380029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] MFA Rules not processed for user `28be028abef24acaa41795a50c267ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:51.381365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:51.507915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1142/2290] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:51.521445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:51.523356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:51.664846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e283b-bd0e-45ac-8c06-79f387a2a1b0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:51.669167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c4faf-9ffd-47ea-a1c1-3bee74714fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:51.669757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c4faf-9ffd-47ea-a1c1-3bee74714fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:51.670376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c4faf-9ffd-47ea-a1c1-3bee74714fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:51.670519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c4faf-9ffd-47ea-a1c1-3bee74714fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:51.687387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c4faf-9ffd-47ea-a1c1-3bee74714fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:51.689273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c4faf-9ffd-47ea-a1c1-3bee74714fb1 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:51.711790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1149/2291] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:52.494259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:52.495733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:52.653092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608762f-5c61-44df-ad8f-8d21b62dc5ac None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:52.659031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:52.660025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:52.660304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:52.660590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] PATH_INFO: `/v3/users/fafefcbb62bf4284a8a9da5d6c0a4609` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:52.662807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafefcbb62bf4284a8a9da5d6c0a4609)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:52.664511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:52.686147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:52.767028 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafefcbb62bf4284a8a9da5d6c0a4609'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:52.796082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafefcbb62bf4284a8a9da5d6c0a4609'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:52.796258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafefcbb62bf4284a8a9da5d6c0a4609'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:52.796837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafefcbb62bf4284a8a9da5d6c0a4609'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:52.811267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafefcbb62bf4284a8a9da5d6c0a4609'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:52.846417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1143/2292] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:52 2022] DELETE /identity/v3/users/fafefcbb62bf4284a8a9da5d6c0a4609 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:52.851139 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:52.851840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:52.851840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:52.902970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:52.905068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:53.091095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1150/2293] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:53.626102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:53.627637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:53.784910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7389420-c327-4c06-b401-c33ee8251aba admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:53.795191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c30218-944d-4190-83f2-79c845a20724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:53.796130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c30218-944d-4190-83f2-79c845a20724 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:53.796796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c30218-944d-4190-83f2-79c845a20724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:53.797296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c30218-944d-4190-83f2-79c845a20724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:53.823286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c30218-944d-4190-83f2-79c845a20724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:53.825140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c30218-944d-4190-83f2-79c845a20724 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:53.849971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1144/2294] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:54.466089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:54.468175 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:54.499982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76924f04-c9da-4c21-85af-b30d5a90bca4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:54.516078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:54.519443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:54.519572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:54.519806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] PATH_INFO: `/v3/projects/894127095d6445f5a2eafc9906ce5cf7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:54.522604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=894127095d6445f5a2eafc9906ce5cf7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:54.525108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:54.545837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:54.611573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894127095d6445f5a2eafc9906ce5cf7', 'request_id': 'req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:54.626536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894127095d6445f5a2eafc9906ce5cf7', 'request_id': 'req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:54.635405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894127095d6445f5a2eafc9906ce5cf7', 'request_id': 'req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:54.656670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] The token cache is being invalidate because project 894127095d6445f5a2eafc9906ce5cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:32:54.657198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:54.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:54.660318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1151/2295] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:32:54 2022] DELETE /identity/v3/projects/894127095d6445f5a2eafc9906ce5cf7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:56.915114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff898af-8d0d-4d04-b001-b2085eeea3ec None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:56.915998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff898af-8d0d-4d04-b001-b2085eeea3ec None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:56.916441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff898af-8d0d-4d04-b001-b2085eeea3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:56.958144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cff898af-8d0d-4d04-b001-b2085eeea3ec None None] MFA Rules not processed for user `9236c3e6183145d0a873b7c59754087b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:56.959643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cff898af-8d0d-4d04-b001-b2085eeea3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:57.086874 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1145/2296] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:57.099181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:57.100756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:57.166467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8d9b7-f67d-4da4-a0e8-531ad33238fd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:57.170656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:57.171308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:57.171667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:57.171943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:57.173897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:57.233398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:57.235002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:57.245972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1152/2297] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:32:57.353885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:57.354244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:57.354588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:57.374041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:57.375261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:57.384628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:57.450594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1146/2298] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:32:57.464403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:57.466280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:57.535965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:57.557689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f2c9-c929-40ad-aeb8-28299968ad8c service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:57.564317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0db2a-03a9-4e57-9abb-45025bfa9787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:57.565033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0db2a-03a9-4e57-9abb-45025bfa9787 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:57.565311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0db2a-03a9-4e57-9abb-45025bfa9787 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:57.565611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0db2a-03a9-4e57-9abb-45025bfa9787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:57.574002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0db2a-03a9-4e57-9abb-45025bfa9787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:57.575892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0db2a-03a9-4e57-9abb-45025bfa9787 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:57.585331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1153/2299] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:32:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:32:59.180953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:59.182505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:59.426533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbb985-ef17-4101-81ae-2c06b8216670 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:32:59.433209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:32:59.434808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:59.435443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:59.436297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] PATH_INFO: `/v3/users/28be028abef24acaa41795a50c267ff3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:59.439440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28be028abef24acaa41795a50c267ff3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:32:59.442505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:32:59.546336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28be028abef24acaa41795a50c267ff3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:59.566924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28be028abef24acaa41795a50c267ff3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:59.568079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28be028abef24acaa41795a50c267ff3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:59.569591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28be028abef24acaa41795a50c267ff3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:59.582369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28be028abef24acaa41795a50c267ff3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:32:59.631107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1147/2300] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:32:59 2022] DELETE /identity/v3/users/28be028abef24acaa41795a50c267ff3 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:32:59.635782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:32:59.636667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:32:59.637096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:32:59.674290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:32:59.675857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:59.811963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1154/2301] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:32:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:32:59.822476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:32:59.824317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:32:59.953050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:32:59.993681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37444-6077-420c-aabe-c269ebfc7005 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:00.006360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcf648-2e09-453a-ae99-7ad1aa150951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:00.007170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcf648-2e09-453a-ae99-7ad1aa150951 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:00.007455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcf648-2e09-453a-ae99-7ad1aa150951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:00.007731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcf648-2e09-453a-ae99-7ad1aa150951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:00.029507 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcf648-2e09-453a-ae99-7ad1aa150951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:00.031404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcf648-2e09-453a-ae99-7ad1aa150951 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:00.061375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1148/2302] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:32:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:00.464720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:00.466998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:00.492712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7f136-4c72-4222-b9b2-4699d0ab7168 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:00.511506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:00.513637 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:00.513932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:00.514604 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] PATH_INFO: `/v3/users/2c1f5a727a1f4f53b97a5ce38df94fab` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:00.518311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1f5a727a1f4f53b97a5ce38df94fab)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:00.524515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:00.644172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1f5a727a1f4f53b97a5ce38df94fab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:00.656297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1f5a727a1f4f53b97a5ce38df94fab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:00.657428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1f5a727a1f4f53b97a5ce38df94fab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:00.658371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1f5a727a1f4f53b97a5ce38df94fab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:00.676967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1f5a727a1f4f53b97a5ce38df94fab'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:00.723741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:00.725117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:00.726442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:00.728166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1155/2303] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:00 2022] DELETE /identity/v3/users/2c1f5a727a1f4f53b97a5ce38df94fab => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:00.759802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:00.761366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:00.945872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1149/2304] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:00.956043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:00.957532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:00.984296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:00.987228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:01.013026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa868b1f-7c97-45c5-b111-aebd7958cc9f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:01.018606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:01.019754 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.020219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.020719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] PATH_INFO: `/v3/projects/47bad3b31c7f48578466b592680befaf` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.023001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bad3b31c7f48578466b592680befaf)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:01.027001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:01.097992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d9b7-2ca6-4cc9-bd57-28ad28c3b6bb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:01.098859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bad3b31c7f48578466b592680befaf', 'request_id': 'req-2d692838-8b1c-4694-9b06-0b24c658365a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:01.112621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:01.120057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bad3b31c7f48578466b592680befaf', 'request_id': 'req-2d692838-8b1c-4694-9b06-0b24c658365a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:01.132170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bad3b31c7f48578466b592680befaf', 'request_id': 'req-2d692838-8b1c-4694-9b06-0b24c658365a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:01.160789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] The token cache is being invalidate because project 47bad3b31c7f48578466b592680befaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:01.161147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:01.176177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2d692838-8b1c-4694-9b06-0b24c658365a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:01.178084 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1150/2305] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:00 2022] DELETE /identity/v3/projects/47bad3b31c7f48578466b592680befaf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:01.184909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.186604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.186907 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.221919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:01.222878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.223007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.223203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.223540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:01.224973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:01.225707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:01.321766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:01.323275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330aaa3-92b4-4ffb-9a9c-240657cf4d04 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:01.343225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1156/2306] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:33:01.476359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1151/2307] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:01.486500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.486968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.487302 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.549616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:01.553202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:01.582615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1157/2308] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:01.594006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:01.601973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:01.606760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebec5e-ea9f-4ef4-8375-6990d707501f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:01.613559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:01.614154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.614538 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.615578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.616064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:01.616470 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:01.620305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:01.667551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:01.673957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1152/2309] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:01.678511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16dd4d-e92e-4a78-abd0-6d0f9abce590 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:01.689160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:01.690276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.690819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.691323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.692348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:01.695018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:01.752269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1158/2310] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:01 2022] POST /identity/v3/projects => generated 363 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:01.757031 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:01.770655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71161017-606b-4158-8a8c-527c74be8075 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:01.968867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:01.969333 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:01.969652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:01.969906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:01.973094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:01.979522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.020176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:02.020623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1153/2311] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:01 2022] POST /identity/v3/users => generated 282 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:33:02.035820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea285410-8c03-47ec-9bb7-42fc0f39eeb5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:02.040548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:02.041383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.041692 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.042305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.043040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:02.044739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.102180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:02.112999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1159/2312] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:33:02.115384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ca9bc-8b55-4f4c-af31-53f3d1f3b9d6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:02.122988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:02.131027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:02.153745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:02.154472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.154756 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.155032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.162732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:02.164444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.177442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e0222-6486-45b9-8edc-7c7b81ebb553 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:02.192640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1154/2313] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:02.193640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:02.203346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:02.205009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.205606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.206115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] PATH_INFO: `/v3/projects/c14109d8b99a4ab1ad0008ad888b3140` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.215272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c14109d8b99a4ab1ad0008ad888b3140)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:02.228341 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.260480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdd0be-5c07-45f2-9754-8e2a87d3951f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:02.277672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5401a48-7597-42f4-860d-6551c58574de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:02.278692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401a48-7597-42f4-860d-6551c58574de admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.278971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401a48-7597-42f4-860d-6551c58574de admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.279247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5401a48-7597-42f4-860d-6551c58574de admin admin] PATH_INFO: `/v3/projects/c27c3a7487164a079d01a93d2e7387a4/users/ad09c35954de4a4a98bcdd9741dd1c37/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.334337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5401a48-7597-42f4-860d-6551c58574de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27c3a7487164a079d01a93d2e7387a4, user_id=ad09c35954de4a4a98bcdd9741dd1c37, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:02.340146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5401a48-7597-42f4-860d-6551c58574de admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.348080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c14109d8b99a4ab1ad0008ad888b3140', 'request_id': 'req-7d0184d8-f84c-45de-b580-7d8e950b7e9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.365330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c14109d8b99a4ab1ad0008ad888b3140', 'request_id': 'req-7d0184d8-f84c-45de-b580-7d8e950b7e9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.371354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1155/2314] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:02 2022] PUT /identity/v3/projects/c27c3a7487164a079d01a93d2e7387a4/users/ad09c35954de4a4a98bcdd9741dd1c37/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:02.381761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c14109d8b99a4ab1ad0008ad888b3140', 'request_id': 'req-7d0184d8-f84c-45de-b580-7d8e950b7e9e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.386332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.386698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.386994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.405437 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] The token cache is being invalidate because project c14109d8b99a4ab1ad0008ad888b3140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:02.406062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.407404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.409531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1160/2315] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:02 2022] DELETE /identity/v3/projects/c14109d8b99a4ab1ad0008ad888b3140 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:02.414830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:02.418226 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:02.438518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:02.451779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] MFA Rules not processed for user `ad09c35954de4a4a98bcdd9741dd1c37`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:02.453106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:02.568860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0184d8-f84c-45de-b580-7d8e950b7e9e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:02.607369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1156/2316] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:02.616447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:02.620008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:02.729223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:02.758204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fc073-50f1-4b35-a0b1-8ce0b49a8f44 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:02.764619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:02.765925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:02.766435 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.766734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.766862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:02.767105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:02.767273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.767386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] PATH_INFO: `/v3/users/dcaae4ff29364c8289eca12d0983f5e0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:02.768785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcaae4ff29364c8289eca12d0983f5e0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:02.779861 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.790467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:02.793669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:02.828294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1157/2317] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:02.897828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcaae4ff29364c8289eca12d0983f5e0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.914806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcaae4ff29364c8289eca12d0983f5e0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.915659 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcaae4ff29364c8289eca12d0983f5e0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.916442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcaae4ff29364c8289eca12d0983f5e0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.945445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a57064d9-8c5f-4b75-a048-9e865e5e9bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcaae4ff29364c8289eca12d0983f5e0'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:02.963694 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1161/2318] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:02 2022] DELETE /identity/v3/users/dcaae4ff29364c8289eca12d0983f5e0 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:03.402294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:03.424454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac66f9c-6e67-4b0d-961f-d203f66ac711 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:03.564006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:03.565020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:03.565309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:03.566752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] PATH_INFO: `/v3/projects/9ea73ab493de410aa4ee3a8aa325ea06` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:03.569945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea73ab493de410aa4ee3a8aa325ea06)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:03.573836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:03.658209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea73ab493de410aa4ee3a8aa325ea06', 'request_id': 'req-00882958-25be-4188-8c17-4ba958771104'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:03.673961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea73ab493de410aa4ee3a8aa325ea06', 'request_id': 'req-00882958-25be-4188-8c17-4ba958771104'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:03.682269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea73ab493de410aa4ee3a8aa325ea06', 'request_id': 'req-00882958-25be-4188-8c17-4ba958771104'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:03.704656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] The token cache is being invalidate because project 9ea73ab493de410aa4ee3a8aa325ea06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:03.704951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:03.705789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00882958-25be-4188-8c17-4ba958771104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:03.707639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1158/2319] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:03 2022] DELETE /identity/v3/projects/9ea73ab493de410aa4ee3a8aa325ea06 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:03.712431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:03.712725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:03.713010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:03.736913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:03.738556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:03.773358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:03.845289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1162/2320] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:03.853212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:03.854648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:03.855141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:03.884127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:03.885846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:03.896646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1159/2321] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:03.900819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:03.913458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6fae4-25c0-4321-8244-c400f85e5f73 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:03.920010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:03.920799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:03.921083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:03.921360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:03.922201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:03.922510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:03.924278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:03.960099 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:03.963204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1163/2322] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:33:03.976439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573c031-1af5-4be9-824f-d5409a5f8f6b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:03.981378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:03.982230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:03.982517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:03.982942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:03.984741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:03.986727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.041055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1160/2323] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:03 2022] POST /identity/v3/projects => generated 387 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:04.045294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.058991 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463b16e-b079-4a51-84fe-e5ffa122881b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.177092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.177976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.178393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.178704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.179479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.181519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.217445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1164/2324] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:04 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:04.218870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.231003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f142a08-04e1-4a1d-a749-b953b1b68ea9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.237035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.237864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.238144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.238419 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.239160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.240839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.253362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1161/2325] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:04.263338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.274315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30fa2b-3c92-43eb-9f18-ebe47e3b7289 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.279471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cc76f-8017-487a-abbe-959a05b94c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.280204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc76f-8017-487a-abbe-959a05b94c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.280495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc76f-8017-487a-abbe-959a05b94c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.280771 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cc76f-8017-487a-abbe-959a05b94c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.281539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc76f-8017-487a-abbe-959a05b94c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.283134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cc76f-8017-487a-abbe-959a05b94c0a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.300729 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1165/2326] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:04.301920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.312779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58430b0-eef4-4f14-bbfb-c34f910c86d3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.317383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.318302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.318602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.318876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] PATH_INFO: `/v3/projects/c5cad84826114c81bc012ef503aa895b/users/6bb6a614be804bdd89dc1ddbd62c293b/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.336961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5cad84826114c81bc012ef503aa895b, user_id=6bb6a614be804bdd89dc1ddbd62c293b, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.339772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.356270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1162/2327] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:04 2022] PUT /identity/v3/projects/c5cad84826114c81bc012ef503aa895b/users/6bb6a614be804bdd89dc1ddbd62c293b/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:04.367247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.367966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.368626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.419021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] MFA Rules not processed for user `6bb6a614be804bdd89dc1ddbd62c293b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:04.420388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:04.506931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1166/2328] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:04.508170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.515473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:04.598765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054302f2-a9ba-4faf-a9ed-b146c822d53f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.603947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.604705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.605058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.605342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.606327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.608459 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.645335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1163/2329] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:04 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:33:04.649510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.662155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98397f2-b103-4de6-ab2b-4dfb1c49e3eb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.686703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:04.742264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.742993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.743285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.743551 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.744305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.746597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.783707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1167/2330] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:04 2022] POST /identity/v3/users => generated 300 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:04.785887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.799527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc17255-b843-4d4d-8bdc-f37cc71e5c80 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.804820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.805956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.806227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.806513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.807343 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.808975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.843391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1164/2331] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:33:04.847651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.859665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02600e0-3e95-4273-8e15-2ac5f5700589 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:04.867937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:04.869603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:04.870137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:04.870510 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] PATH_INFO: `/v3/projects/9323b5f277274e4e802657888c1e45ab/users/58c12c6a047644faa0e8807c2b21bd7a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:04.897070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9323b5f277274e4e802657888c1e45ab, user_id=58c12c6a047644faa0e8807c2b21bd7a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:04.900003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:04.915635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1168/2332] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:04 2022] PUT /identity/v3/projects/9323b5f277274e4e802657888c1e45ab/users/58c12c6a047644faa0e8807c2b21bd7a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:04.919859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:04.930522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be09ed-8e1d-42c1-8566-2001eea06d65 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:05.002621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467eb8e-f8cd-4676-b1d6-a9f494b4cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:05.003300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467eb8e-f8cd-4676-b1d6-a9f494b4cf9f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:05.003581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467eb8e-f8cd-4676-b1d6-a9f494b4cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:05.003864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467eb8e-f8cd-4676-b1d6-a9f494b4cf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:05.004596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467eb8e-f8cd-4676-b1d6-a9f494b4cf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:05.006250 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467eb8e-f8cd-4676-b1d6-a9f494b4cf9f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:05.018388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1165/2333] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:05.022623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:05.034565 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c095-b767-4fe5-995b-c5bc71cf5fed admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:05.039793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:05.040691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:05.040969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:05.041237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] PATH_INFO: `/v3/projects/9323b5f277274e4e802657888c1e45ab/users/58c12c6a047644faa0e8807c2b21bd7a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:05.043628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9323b5f277274e4e802657888c1e45ab, user_id=58c12c6a047644faa0e8807c2b21bd7a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:05.046884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:05.069089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1169/2334] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:05 2022] PUT /identity/v3/projects/9323b5f277274e4e802657888c1e45ab/users/58c12c6a047644faa0e8807c2b21bd7a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:05.071365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:05.071661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:05.071949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:05.103984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] MFA Rules not processed for user `58c12c6a047644faa0e8807c2b21bd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:05.105136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:05.191563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:05.204170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1166/2335] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:05.214408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:05.216324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:05.297440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c2dc2-ca90-4680-8c3b-6261ceedd8b2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:05.303459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97370654-657f-4164-96e3-6f3ad2d86c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:05.304212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370654-657f-4164-96e3-6f3ad2d86c47 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:05.308172 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370654-657f-4164-96e3-6f3ad2d86c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:05.308566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97370654-657f-4164-96e3-6f3ad2d86c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:05.329350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97370654-657f-4164-96e3-6f3ad2d86c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:05.330888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97370654-657f-4164-96e3-6f3ad2d86c47 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:05.344619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1170/2336] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:05.569292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:05.681706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b7d8e-e12d-4635-a45c-ddbe0e7dc295 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:05.692842 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:05.693786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:05.694695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:05.694695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] PATH_INFO: `/v3/users/6bb6a614be804bdd89dc1ddbd62c293b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:05.696971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb6a614be804bdd89dc1ddbd62c293b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:05.699229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:05.819467 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb6a614be804bdd89dc1ddbd62c293b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:05.849405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb6a614be804bdd89dc1ddbd62c293b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:05.857711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb6a614be804bdd89dc1ddbd62c293b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:05.870570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb6a614be804bdd89dc1ddbd62c293b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:05.883837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb6a614be804bdd89dc1ddbd62c293b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:05.915509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1167/2337] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:05 2022] DELETE /identity/v3/users/6bb6a614be804bdd89dc1ddbd62c293b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:05.931543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:05.932133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:05.932250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:05.958547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:05.959815 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:06.039911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1171/2338] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:06.054616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:06.056078 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:06.145347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7e35-c05f-4d3b-ae57-a26382dd63d6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:06.149712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:06.150373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:06.150662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:06.150967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:06.165663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:06.167733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:06.179758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1168/2339] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:06.650994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:06.652456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:06.697074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbeffe-d499-438e-8f5b-6c57702bb474 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:06.718839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:06.719718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:06.719995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:06.720268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] PATH_INFO: `/v3/projects/c5cad84826114c81bc012ef503aa895b` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:06.724183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5cad84826114c81bc012ef503aa895b)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:06.726484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:06.774410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:06.827649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5cad84826114c81bc012ef503aa895b', 'request_id': 'req-70f82e46-5a1d-49db-9efc-1a14523d9abc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:06.838834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5cad84826114c81bc012ef503aa895b', 'request_id': 'req-70f82e46-5a1d-49db-9efc-1a14523d9abc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:06.848855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5cad84826114c81bc012ef503aa895b', 'request_id': 'req-70f82e46-5a1d-49db-9efc-1a14523d9abc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:06.865355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] The token cache is being invalidate because project c5cad84826114c81bc012ef503aa895b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:06.865790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:06.866446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:06.868256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1172/2340] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:06 2022] DELETE /identity/v3/projects/c5cad84826114c81bc012ef503aa895b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:06.872254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:06.884952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c58934-a186-4042-ae57-5fba51624639 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:06.922816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:06.977373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:06.978297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:06.978582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:06.978863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] PATH_INFO: `/v3/users/58c12c6a047644faa0e8807c2b21bd7a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:06.980114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c12c6a047644faa0e8807c2b21bd7a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:06.982976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:07.062548 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c12c6a047644faa0e8807c2b21bd7a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.079796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c12c6a047644faa0e8807c2b21bd7a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.080533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c12c6a047644faa0e8807c2b21bd7a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.081142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c12c6a047644faa0e8807c2b21bd7a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.089135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f7fbbebb-1db1-459c-ade4-5386755adcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c12c6a047644faa0e8807c2b21bd7a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.112519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1169/2341] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:06 2022] DELETE /identity/v3/users/58c12c6a047644faa0e8807c2b21bd7a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:07.522459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:07.524180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:07.543498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f82e46-5a1d-49db-9efc-1a14523d9abc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:07.628905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:07.630053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:07.630478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:07.630909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] PATH_INFO: `/v3/projects/9323b5f277274e4e802657888c1e45ab` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:07.632271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9323b5f277274e4e802657888c1e45ab)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:07.634456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:07.719400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9323b5f277274e4e802657888c1e45ab', 'request_id': 'req-5e93b29e-5da0-4c50-b1de-3b457abab671'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.729234 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9323b5f277274e4e802657888c1e45ab', 'request_id': 'req-5e93b29e-5da0-4c50-b1de-3b457abab671'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.736664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9323b5f277274e4e802657888c1e45ab', 'request_id': 'req-5e93b29e-5da0-4c50-b1de-3b457abab671'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.751361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] The token cache is being invalidate because project 9323b5f277274e4e802657888c1e45ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:07.751655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.752332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5e93b29e-5da0-4c50-b1de-3b457abab671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:07.757606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1173/2342] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:07 2022] DELETE /identity/v3/projects/9323b5f277274e4e802657888c1e45ab => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:07.761717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:07.761717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:07.762209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:07.783071 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:07.784316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:07.931242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1170/2343] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:07.931717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:07.932010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:07.932285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:07.970110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:07.973548 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:07.983863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1174/2344] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:07.990584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.020241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce87af8-a693-4663-b395-acce0632cc70 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.027118 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.027797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.028285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.028709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.029496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:08.030549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.032515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.069820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1171/2345] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:08.071640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.085247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa66a7-c991-4c01-bcee-0724053c83f8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.089982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.090727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.091016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.091450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.092154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.094337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.137482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1175/2346] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:08 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:33:08.141049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.155175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19f335-a4de-49c0-9c2e-7168a797fc16 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.188798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:08.224322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.225108 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.225471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.225971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.226747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.228744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.273024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1172/2347] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:08 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:08.273941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.282873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:08.290312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651828d8-1207-441a-b5a6-7c743a014bbb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.309761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.310663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.313114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.313527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.314858 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.316598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.343961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1176/2348] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:08.345465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.356366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc7a28-3152-448e-8a3a-3eb0332ab0eb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.361700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861dc54-4a86-4e97-8bc1-2b6cfa8d85de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.362372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861dc54-4a86-4e97-8bc1-2b6cfa8d85de admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.362654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861dc54-4a86-4e97-8bc1-2b6cfa8d85de admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.362927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861dc54-4a86-4e97-8bc1-2b6cfa8d85de admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.363735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861dc54-4a86-4e97-8bc1-2b6cfa8d85de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.365554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861dc54-4a86-4e97-8bc1-2b6cfa8d85de admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.385289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1173/2349] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:08.389907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.401052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bca7ea-ecb5-4920-95ac-575d4facf7f6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.410820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.412026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.412407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.412783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] PATH_INFO: `/v3/projects/2fab2ecf969047fbbd120b8dfc3d4bad/users/21ba070e71a24f68bf2788745581c93b/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.433943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fab2ecf969047fbbd120b8dfc3d4bad, user_id=21ba070e71a24f68bf2788745581c93b, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.437269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.464840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1177/2350] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:08 2022] PUT /identity/v3/projects/2fab2ecf969047fbbd120b8dfc3d4bad/users/21ba070e71a24f68bf2788745581c93b/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:08.478763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.479023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.479313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.515759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] MFA Rules not processed for user `21ba070e71a24f68bf2788745581c93b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:08.516977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:08.631027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1174/2351] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:08.635843 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.749251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67809b-f8ac-4ca7-94a5-bc47c1f2ce20 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.754572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.755314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.755664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.756021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.756782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.758970 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.796740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1178/2352] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:08 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:08.801076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:08.816681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d1ef7-654b-43eb-96d5-8d4cccd2b5a6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:08.926749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:08.927653 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:08.928096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:08.930411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:08.931389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:08.934415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:08.975355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1175/2353] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:08 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:33:08.978844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:09.003799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394946dc-bc81-431c-bcaf-2efb2cbea57d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:09.015066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:09.016059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:09.016400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:09.016722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:09.018405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:09.020268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:09.035373 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1179/2354] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:09.039818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:09.059352 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc69ae-87e7-4dcb-aff2-34b554528f1c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:09.073502 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:09.074884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:09.075486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:09.075658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] PATH_INFO: `/v3/projects/c65be2983cb347748f0b6a6deb6e6536/users/34998dbae8f24ee3ba900037757c97a5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:09.097667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65be2983cb347748f0b6a6deb6e6536, user_id=34998dbae8f24ee3ba900037757c97a5, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:09.106693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:09.130724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1176/2355] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:09 2022] PUT /identity/v3/projects/c65be2983cb347748f0b6a6deb6e6536/users/34998dbae8f24ee3ba900037757c97a5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:33:09.135000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:09.151197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ff63-5e47-4bbb-bd7a-58fec1dff0f4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:09.347477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968b460-a91e-40fd-b876-729119b410da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:09.348218 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b460-a91e-40fd-b876-729119b410da admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:09.348595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b460-a91e-40fd-b876-729119b410da admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:09.348884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968b460-a91e-40fd-b876-729119b410da admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:09.349674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968b460-a91e-40fd-b876-729119b410da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:09.351504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968b460-a91e-40fd-b876-729119b410da admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:09.369065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1180/2356] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:09.370358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:09.376312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:09.387309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290f3db-79af-4117-95d8-49e9adf4f750 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:09.394610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:09.397927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:09.398325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:09.398709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] PATH_INFO: `/v3/projects/c65be2983cb347748f0b6a6deb6e6536/users/34998dbae8f24ee3ba900037757c97a5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:09.403963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65be2983cb347748f0b6a6deb6e6536, user_id=34998dbae8f24ee3ba900037757c97a5, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:09.414290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:09.422596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1177/2357] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:09 2022] PUT /identity/v3/projects/c65be2983cb347748f0b6a6deb6e6536/users/34998dbae8f24ee3ba900037757c97a5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:09.433141 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806ba5e-68f3-4173-8a4f-012e06ea0242 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:09.433489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806ba5e-68f3-4173-8a4f-012e06ea0242 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:09.433741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806ba5e-68f3-4173-8a4f-012e06ea0242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:09.470061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-2806ba5e-68f3-4173-8a4f-012e06ea0242 None None] MFA Rules not processed for user `34998dbae8f24ee3ba900037757c97a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:09.471479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-2806ba5e-68f3-4173-8a4f-012e06ea0242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:09.512096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:09.573800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1181/2358] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:09.583160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:09.585651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:09.668329 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b89dd-c8ba-4024-a7f1-937b70201a6e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:09.673079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:09.673786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:09.674096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:09.674375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:09.684950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:09.697526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:09.699835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1178/2359] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:10.381150 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a21ca-829c-49b4-ad67-8ffea051d3c2 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:10.381844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a21ca-829c-49b4-ad67-8ffea051d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:10.382128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a21ca-829c-49b4-ad67-8ffea051d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:10.406182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-766a21ca-829c-49b4-ad67-8ffea051d3c2 None None] MFA Rules not processed for user `d9a8a480afb3438cb269ebd710c78018`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:10.407509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-766a21ca-829c-49b4-ad67-8ffea051d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:10.520025 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1182/2360] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:10.571967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:10.674311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8d0ba-f5dc-4a33-87ee-37d292922b99 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:10.680088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:10.680998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:10.681301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:10.681611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] PATH_INFO: `/v3/users/21ba070e71a24f68bf2788745581c93b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:10.682828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ba070e71a24f68bf2788745581c93b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:10.684801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:10.696423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:10.696765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:10.696990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:10.726278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] MFA Rules not processed for user `b11df35c0e4541deaea0494759301a94`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:10.727282 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:10.748023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ba070e71a24f68bf2788745581c93b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:10.769215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ba070e71a24f68bf2788745581c93b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:10.770160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ba070e71a24f68bf2788745581c93b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:10.771079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ba070e71a24f68bf2788745581c93b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:10.781791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20c22230-ad5c-4a2d-9216-212889908589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ba070e71a24f68bf2788745581c93b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:10.801020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1179/2361] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:10 2022] DELETE /identity/v3/users/21ba070e71a24f68bf2788745581c93b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:10.804334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1183/2362] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:10.805317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:10.805633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:10.805913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:10.826309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:10.827516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:10.833401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:10.897449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1180/2363] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:10.911559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:10.913647 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:10.984931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6e6f0-b8cd-4232-aa2b-dd5a01524071 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:11.030335 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:11.060935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:11.061738 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:11.062246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:11.062617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:11.074244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:11.076463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:11.090760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1184/2364] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:11.570796 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:11.572102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:11.588165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461224d4-5b18-428f-b019-013bd542bd97 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:11.596222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:11.598423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:11.598935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:11.599404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] PATH_INFO: `/v3/users/3503c3ce6d374a20926d7312859eade6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:11.601443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3503c3ce6d374a20926d7312859eade6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:11.604935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:11.662269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:11.665340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:11.688917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0f240-edc8-4e76-8564-02bdb276f803 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:11.701797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:11.703569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:11.704199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:11.704879 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] PATH_INFO: `/v3/projects/2fab2ecf969047fbbd120b8dfc3d4bad` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:11.706640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fab2ecf969047fbbd120b8dfc3d4bad)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:11.709203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:11.733511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3503c3ce6d374a20926d7312859eade6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.753180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3503c3ce6d374a20926d7312859eade6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.753981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3503c3ce6d374a20926d7312859eade6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.754590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3503c3ce6d374a20926d7312859eade6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.764700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-effa4d4c-0726-48e1-9991-0a1f4651c05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3503c3ce6d374a20926d7312859eade6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.781979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1181/2365] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:11 2022] DELETE /identity/v3/users/3503c3ce6d374a20926d7312859eade6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:11.787396 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:11.788019 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:11.788266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:11.797571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fab2ecf969047fbbd120b8dfc3d4bad', 'request_id': 'req-4b12f188-3958-4f7c-9952-413f602fdbb4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.825998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fab2ecf969047fbbd120b8dfc3d4bad', 'request_id': 'req-4b12f188-3958-4f7c-9952-413f602fdbb4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.831535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:11.834789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:11.845254 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fab2ecf969047fbbd120b8dfc3d4bad', 'request_id': 'req-4b12f188-3958-4f7c-9952-413f602fdbb4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.867102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] The token cache is being invalidate because project 2fab2ecf969047fbbd120b8dfc3d4bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:11.867482 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.868522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:11.870474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1185/2366] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:11 2022] DELETE /identity/v3/projects/2fab2ecf969047fbbd120b8dfc3d4bad => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:33:11.878455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:11.880070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:11.951915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1182/2367] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:11.974411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:11.976360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:11.976833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12f188-3958-4f7c-9952-413f602fdbb4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:11.982449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:11.983424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:11.983842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:11.984196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] PATH_INFO: `/v3/users/34998dbae8f24ee3ba900037757c97a5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:11.985515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=34998dbae8f24ee3ba900037757c97a5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:11.987721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:12.018336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:12.054059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34998dbae8f24ee3ba900037757c97a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.065556 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34998dbae8f24ee3ba900037757c97a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.066198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34998dbae8f24ee3ba900037757c97a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.066844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34998dbae8f24ee3ba900037757c97a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.070805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45499708-dfa5-47cb-b223-654f72adb92c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:12.072107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:12.077137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:12.078178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.078500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.078890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.080765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.083128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34998dbae8f24ee3ba900037757c97a5'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.094167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:12.095746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:12.099330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1186/2368] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:11 2022] DELETE /identity/v3/users/34998dbae8f24ee3ba900037757c97a5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:12.123746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1183/2369] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:33:12.418624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:12.420489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.488135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:12.490043 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.517517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301406fc-9eb4-442b-a2ab-83d216089ec2 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:12.523180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:12.524553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.524986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.525380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] PATH_INFO: `/v3/projects/83eb6b286ea74e0eaaece425cc668ace` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.528181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83eb6b286ea74e0eaaece425cc668ace)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:12.530861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:12.563956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cee5ab-8c83-4ab6-b407-adc87f46ef1f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:12.576584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:12.577580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.577882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.578212 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] PATH_INFO: `/v3/projects/c65be2983cb347748f0b6a6deb6e6536` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.579621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65be2983cb347748f0b6a6deb6e6536)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:12.581753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:12.600226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83eb6b286ea74e0eaaece425cc668ace', 'request_id': 'req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.611724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83eb6b286ea74e0eaaece425cc668ace', 'request_id': 'req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.621935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83eb6b286ea74e0eaaece425cc668ace', 'request_id': 'req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.636485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] The token cache is being invalidate because project 83eb6b286ea74e0eaaece425cc668ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:12.636776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.637707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dc22f3b1-e269-4ddf-baf4-8e4ee55f4065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.641697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1184/2370] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:12 2022] DELETE /identity/v3/projects/83eb6b286ea74e0eaaece425cc668ace => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:33:12.658560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202038d-fde8-42d8-91ea-322b54e575b5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.658560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202038d-fde8-42d8-91ea-322b54e575b5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.658862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202038d-fde8-42d8-91ea-322b54e575b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.691883 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5202038d-fde8-42d8-91ea-322b54e575b5 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:12.693577 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5202038d-fde8-42d8-91ea-322b54e575b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.707974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65be2983cb347748f0b6a6deb6e6536', 'request_id': 'req-18121084-3382-4a04-8b98-c792027c7156'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.725260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65be2983cb347748f0b6a6deb6e6536', 'request_id': 'req-18121084-3382-4a04-8b98-c792027c7156'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.742841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65be2983cb347748f0b6a6deb6e6536', 'request_id': 'req-18121084-3382-4a04-8b98-c792027c7156'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.775084 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] The token cache is being invalidate because project c65be2983cb347748f0b6a6deb6e6536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:12.775388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.779550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-18121084-3382-4a04-8b98-c792027c7156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:12.783829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1187/2371] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:12 2022] DELETE /identity/v3/projects/c65be2983cb347748f0b6a6deb6e6536 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:12.788425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5787c-f3d7-48fa-b416-1f3065956a0c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.788814 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5787c-f3d7-48fa-b416-1f3065956a0c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.789399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5787c-f3d7-48fa-b416-1f3065956a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.811326 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1185/2372] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:12.826782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.827240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.828195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.834221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-dac5787c-f3d7-48fa-b416-1f3065956a0c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:12.839357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-dac5787c-f3d7-48fa-b416-1f3065956a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.861429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1188/2373] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:12.870423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04998092-0d56-4764-9652-8a10287d80dc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.870941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04998092-0d56-4764-9652-8a10287d80dc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.871370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04998092-0d56-4764-9652-8a10287d80dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.891833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:12.895858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.911511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-04998092-0d56-4764-9652-8a10287d80dc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:12.914941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-04998092-0d56-4764-9652-8a10287d80dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:12.917397 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1186/2374] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:12.928854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:12.939422 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1189/2375] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:12.943251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be8735-25a6-496b-890a-ab31a58f6a2b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:12.943635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04998092-0d56-4764-9652-8a10287d80dc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:12.954463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:12.955550 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.955830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.956109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.956898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:12.957224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:12.959063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:12.962875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04998092-0d56-4764-9652-8a10287d80dc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:12.968719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:12.969710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:12.970188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:12.970638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:12.971587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:12.972087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:12.973929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:12.975242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1187/2376] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:12.979403 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.004082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab237a-9f62-4890-88b3-ddbed85439b0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.018289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1190/2377] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:13.021105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.021720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.022508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.022851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.023194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.024218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.027739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.044513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b4631-7f06-4a0b-b10e-7c0ee04eba20 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.060306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.061100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.061612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.061890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.062680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.064591 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.071773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1188/2378] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:12 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:33:13.076153 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.094956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:13.118044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1191/2379] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:13 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:13.122933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.160403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:13.205475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149de1-a310-43ea-bd42-52262125fc8b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.229239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b4929-d932-4e26-8e4b-c703655d51fa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.235098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.235790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.236389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.236669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.237450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.239383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.270691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1192/2380] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:13 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:33:13.274701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.287475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edd50-536e-4875-8889-6d6101e4ddb7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.293431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.294423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.294901 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.295191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.295939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.297665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.305350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.306060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.306344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.306622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.307334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.309521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.313642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1193/2381] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:13.315976 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.328937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be60f66-6fdf-473c-9cdf-99b8ca259f74 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.334977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.335660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.335941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.336259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.336773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1189/2382] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:13 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:13.340105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.341180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.342463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.354361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c76e-4ea5-4ded-a577-80fb146f868f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.354572 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1194/2383] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:13.358825 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.360313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.360994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.361274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.361843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.362582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.364335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.373538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2760e0b-1116-44b8-b85d-395793d05d14 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.379256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1190/2384] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:33:13.380765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.381968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561dae6-8b19-42d8-9f16-8ab58285a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.383730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561dae6-8b19-42d8-9f16-8ab58285a8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.383730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561dae6-8b19-42d8-9f16-8ab58285a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.383730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561dae6-8b19-42d8-9f16-8ab58285a8b6 admin admin] PATH_INFO: `/v3/projects/02fd5a2e6839481fa805281b6ddaef94/users/f4ce2fad2bde4d639fb1cccdf1461c1b/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.392171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435080ef-6a9d-439d-9ccf-0e873ee52213 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.398438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.398764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.399172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.399456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.400225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.401683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561dae6-8b19-42d8-9f16-8ab58285a8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fd5a2e6839481fa805281b6ddaef94, user_id=f4ce2fad2bde4d639fb1cccdf1461c1b, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.403017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.405085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561dae6-8b19-42d8-9f16-8ab58285a8b6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.418878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1191/2385] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:13.419979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.431692 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45657e-f12c-4be8-8fce-59643746a07f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.434010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1195/2386] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:13 2022] PUT /identity/v3/projects/02fd5a2e6839481fa805281b6ddaef94/users/f4ce2fad2bde4d639fb1cccdf1461c1b/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:33:13.435725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.435725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.435725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.463235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] MFA Rules not processed for user `f4ce2fad2bde4d639fb1cccdf1461c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:13.464630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:13.502753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf06856-fe76-4ad9-a9ae-447895e465a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.503650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf06856-fe76-4ad9-a9ae-447895e465a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.503928 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf06856-fe76-4ad9-a9ae-447895e465a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.504199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf06856-fe76-4ad9-a9ae-447895e465a4 admin admin] PATH_INFO: `/v3/projects/17a97caee8204a65978f6b7d81716224/users/a69f739623174af0a6cb4d066b1caabf/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.525095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf06856-fe76-4ad9-a9ae-447895e465a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a97caee8204a65978f6b7d81716224, user_id=a69f739623174af0a6cb4d066b1caabf, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.528949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf06856-fe76-4ad9-a9ae-447895e465a4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.548757 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1192/2387] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:13 2022] PUT /identity/v3/projects/17a97caee8204a65978f6b7d81716224/users/a69f739623174af0a6cb4d066b1caabf/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:13.549237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1196/2388] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:13.559234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.560126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.560126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.566773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.569162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:13.572091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:13.593690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] MFA Rules not processed for user `a69f739623174af0a6cb4d066b1caabf`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:13.595121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:13.646046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6dd0a2-6f80-476b-b9a6-60bee3d529e8 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.650880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:13.679244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1193/2389] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:13.680140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.681566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:13.700868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c35f17-5fd7-4afb-817a-3d87a3dc8a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.701563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c35f17-5fd7-4afb-817a-3d87a3dc8a11 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.701842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c35f17-5fd7-4afb-817a-3d87a3dc8a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.702116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c35f17-5fd7-4afb-817a-3d87a3dc8a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.743885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f414df4-02a1-4eb1-b0a3-c8fbed56b721 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.747877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.748533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.748821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.749151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.759284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.760762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.770962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1194/2390] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:13.776939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c35f17-5fd7-4afb-817a-3d87a3dc8a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.778721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c35f17-5fd7-4afb-817a-3d87a3dc8a11 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.790787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1197/2391] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:33:13.812929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:13.917383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6932264-c501-4825-b6a3-f7bd1545d6ea service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:13.925462 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:13.925973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:13.926251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:13.926522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] PATH_INFO: `/v3/users/f4ce2fad2bde4d639fb1cccdf1461c1b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:13.928196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ce2fad2bde4d639fb1cccdf1461c1b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:13.930881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:13.986100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:13.993386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ce2fad2bde4d639fb1cccdf1461c1b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.009196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ce2fad2bde4d639fb1cccdf1461c1b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.010365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ce2fad2bde4d639fb1cccdf1461c1b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.012114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ce2fad2bde4d639fb1cccdf1461c1b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.023766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ce2fad2bde4d639fb1cccdf1461c1b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.040726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1195/2392] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:13 2022] DELETE /identity/v3/users/f4ce2fad2bde4d639fb1cccdf1461c1b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:14.046597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:14.047128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:14.047652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:14.094576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:14.095813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:14.180812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1198/2393] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:14.190938 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:14.192426 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:14.273144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ea853-3bd1-4fb9-8e65-95b6d78bd344 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:14.276711 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846cb7-f13b-4b91-8360-d2d59eaa8951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:14.277357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846cb7-f13b-4b91-8360-d2d59eaa8951 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:14.277673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846cb7-f13b-4b91-8360-d2d59eaa8951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:14.278268 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846cb7-f13b-4b91-8360-d2d59eaa8951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:14.288288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846cb7-f13b-4b91-8360-d2d59eaa8951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:14.289786 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846cb7-f13b-4b91-8360-d2d59eaa8951 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:14.300463 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1196/2394] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:14.638114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:14.639847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:14.714946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:14.721290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f607b56-9d3e-4860-b669-1739cc662c5b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:14.726472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:14.727340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:14.727621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:14.727893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] PATH_INFO: `/v3/projects/02fd5a2e6839481fa805281b6ddaef94` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:14.729126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fd5a2e6839481fa805281b6ddaef94)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:14.731208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:14.780260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fd5a2e6839481fa805281b6ddaef94', 'request_id': 'req-e302e46f-8c58-44c8-8e75-2c870c540aaf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.788315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fd5a2e6839481fa805281b6ddaef94', 'request_id': 'req-e302e46f-8c58-44c8-8e75-2c870c540aaf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.794516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fd5a2e6839481fa805281b6ddaef94', 'request_id': 'req-e302e46f-8c58-44c8-8e75-2c870c540aaf'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.806010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] The token cache is being invalidate because project 02fd5a2e6839481fa805281b6ddaef94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:14.806301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.807023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-e302e46f-8c58-44c8-8e75-2c870c540aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:14.808746 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1199/2395] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:14 2022] DELETE /identity/v3/projects/02fd5a2e6839481fa805281b6ddaef94 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:14.814422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:14.814920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:14.815018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:14.864007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:14.865208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:14.931147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1197/2396] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:14.939369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:14.939657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:14.939932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:14.959589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:14.960801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:14.969544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1200/2397] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:14.973931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:14.984229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4a579-f491-4527-a916-c8ad37b96135 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:14.990120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:14.990775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:14.991047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:14.991325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:14.992042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:14.992345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:14.993976 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.006300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1198/2398] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:15.010665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:15.020550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c5f73-7ad8-4d3e-84ba-85ce531861f6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:15.026205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:15.026856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.027134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.027410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.028116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:15.030066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.059913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1201/2399] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:15 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:15.065301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:15.075529 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9997-efec-414a-917d-55e5096bf1df admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:15.101913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:15.143454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:15.144200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.144490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.144753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.145505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:15.147556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.175902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1199/2400] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:15 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:15.180286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:15.196307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96770e27-f32c-4611-b165-b653fee4b053 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:15.202381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:15.203066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.203351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.203629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.204367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:15.206072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.229059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1202/2401] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:15.230331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:15.240927 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c33b5-4447-4302-b4f4-55257638fb90 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:15.245700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3b035-faf8-47aa-a19f-2a4451a1c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:15.246393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b035-faf8-47aa-a19f-2a4451a1c096 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.247346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b035-faf8-47aa-a19f-2a4451a1c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.247831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3b035-faf8-47aa-a19f-2a4451a1c096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.248572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b035-faf8-47aa-a19f-2a4451a1c096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:15.252615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3b035-faf8-47aa-a19f-2a4451a1c096 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.262408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1200/2402] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:15.266095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:15.278124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b14881-3a84-4084-8729-e8fd7360ebb7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:15.285115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:15.286196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.286616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.286984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] PATH_INFO: `/v3/projects/3ac35d497508475599b3843fca19c15e/users/83f6c2bec232400da2cf79be12a16182/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.307428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac35d497508475599b3843fca19c15e, user_id=83f6c2bec232400da2cf79be12a16182, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:15.310317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.325904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1203/2403] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:15 2022] PUT /identity/v3/projects/3ac35d497508475599b3843fca19c15e/users/83f6c2bec232400da2cf79be12a16182/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:15.337566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681588f2-a341-4a56-8b72-389eb36f0a6e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.338675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681588f2-a341-4a56-8b72-389eb36f0a6e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.339271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681588f2-a341-4a56-8b72-389eb36f0a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.375948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-681588f2-a341-4a56-8b72-389eb36f0a6e None None] MFA Rules not processed for user `83f6c2bec232400da2cf79be12a16182`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:15.377251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-681588f2-a341-4a56-8b72-389eb36f0a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:15.446131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1201/2404] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:15.457817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:15.459632 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:15.536633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356801e-89fa-4511-a47b-e1f57f59a29a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:15.540276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:15.540921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:15.541200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:15.541507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:15.550923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:15.552562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:15.562275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1204/2405] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:17.047001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:17.047549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:17.047549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:17.068447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:17.069719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:17.138643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1202/2406] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:33:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:17.151911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:17.154161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:17.165059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:17.238699 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9affb1d-e75f-4482-8c66-2a4ef297501e service glance] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:17.248376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357713e2-7e18-4b27-93a1-a04742d2692f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:17.248920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357713e2-7e18-4b27-93a1-a04742d2692f service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:17.249275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357713e2-7e18-4b27-93a1-a04742d2692f service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:17.249664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357713e2-7e18-4b27-93a1-a04742d2692f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:17.263023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357713e2-7e18-4b27-93a1-a04742d2692f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:17.265096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357713e2-7e18-4b27-93a1-a04742d2692f service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:17.277963 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1205/2407] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:33:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:33:18.690491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:18.692770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:18.751528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:18.782598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d09708-6270-4f14-af5e-d834bf3afbba None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:18.789213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:18.790340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:18.790705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:18.790866 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] PATH_INFO: `/v3/users/a69f739623174af0a6cb4d066b1caabf` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:18.792478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a69f739623174af0a6cb4d066b1caabf)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:18.795100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:18.851959 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a69f739623174af0a6cb4d066b1caabf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:18.865969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a69f739623174af0a6cb4d066b1caabf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:18.867413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a69f739623174af0a6cb4d066b1caabf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:18.868147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a69f739623174af0a6cb4d066b1caabf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:18.881679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a69f739623174af0a6cb4d066b1caabf'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:18.908236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1203/2408] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:18 2022] DELETE /identity/v3/users/a69f739623174af0a6cb4d066b1caabf => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:18.912796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:18.913323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:18.913403 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:18.945378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:18.946760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:19.028358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1206/2409] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:19.064300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:19.066728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:19.279901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3070b-53a3-406a-b22d-ae263b04b515 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:19.284289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234b9e5-bf84-403a-97eb-bfc938912530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:19.293812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234b9e5-bf84-403a-97eb-bfc938912530 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:19.294332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234b9e5-bf84-403a-97eb-bfc938912530 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:19.294889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234b9e5-bf84-403a-97eb-bfc938912530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:19.314535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234b9e5-bf84-403a-97eb-bfc938912530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:19.316871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234b9e5-bf84-403a-97eb-bfc938912530 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:19.331167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1204/2410] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:19.888560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:19.890618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:19.909265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bd5df-34c5-4051-9fe5-67a9f8dc8d67 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:19.945252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:19.945827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:19.946287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:19.946570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] PATH_INFO: `/v3/projects/17a97caee8204a65978f6b7d81716224` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:19.948007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a97caee8204a65978f6b7d81716224)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:19.950073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.013804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a97caee8204a65978f6b7d81716224', 'request_id': 'req-dc2f9082-20c0-4f41-ae02-f365dd5509de'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:20.023380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a97caee8204a65978f6b7d81716224', 'request_id': 'req-dc2f9082-20c0-4f41-ae02-f365dd5509de'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:20.031844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a97caee8204a65978f6b7d81716224', 'request_id': 'req-dc2f9082-20c0-4f41-ae02-f365dd5509de'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:20.054636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] The token cache is being invalidate because project 17a97caee8204a65978f6b7d81716224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:20.055018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:20.055741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-dc2f9082-20c0-4f41-ae02-f365dd5509de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:20.057683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1207/2411] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:19 2022] DELETE /identity/v3/projects/17a97caee8204a65978f6b7d81716224 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:20.063398 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.063973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.064074 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.090604 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:20.092262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:20.153120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:20.190865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1205/2412] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:20.193262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.193749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.194125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.218888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:20.220305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:20.249321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1208/2413] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:20.254035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.272169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c12a3c-81bb-4075-8197-af17aa17fbdc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:20.278766 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:20.279858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.280220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.280535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.281258 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:20.281596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:20.285286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.313292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.319962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1206/2414] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:20.340397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffd0ac-1baa-419e-8deb-8d54572d16ed None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:20.347314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:20.347985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.348265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.348537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.349271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:20.351160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.375552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:20.400663 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1209/2415] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:20 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:20.404779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.417761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158107b-0d60-4c2d-bb40-6b24f2f11b70 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:20.498438 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:20.499170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.499456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.499780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.500443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:20.502968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.532215 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1207/2416] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:20 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:20.533912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.551073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ea7eb-e067-4c9e-bad1-5c52797b47d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:20.556269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:20.556957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.557313 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.557633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.558460 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:20.560455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.579092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1210/2417] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:20.583178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.597365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62b8c0-b259-4259-82a6-1d7c32f116b1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:20.604602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69161c68-052f-44ff-b7e5-6b745fda8d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:20.605385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161c68-052f-44ff-b7e5-6b745fda8d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.605785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161c68-052f-44ff-b7e5-6b745fda8d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.606157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69161c68-052f-44ff-b7e5-6b745fda8d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.606988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69161c68-052f-44ff-b7e5-6b745fda8d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:20.608682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69161c68-052f-44ff-b7e5-6b745fda8d01 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.622506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1208/2418] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:20.626653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.639129 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2521c76-f08d-47b5-99a8-1138681f9dbb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:20.649145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:20.650753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.650979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.651332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] PATH_INFO: `/v3/projects/c96a605108ec47a081c60bc8dc16fd8e/users/d26c50b54171491b987ca844091c6e73/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.672126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96a605108ec47a081c60bc8dc16fd8e, user_id=d26c50b54171491b987ca844091c6e73, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:20.692712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:20.728509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1211/2419] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:20 2022] PUT /identity/v3/projects/c96a605108ec47a081c60bc8dc16fd8e/users/d26c50b54171491b987ca844091c6e73/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:20.733829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bed35-4de7-4ab3-8541-e4dee0ea1007 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:20.734454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bed35-4de7-4ab3-8541-e4dee0ea1007 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:20.735053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bed35-4de7-4ab3-8541-e4dee0ea1007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:20.772695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-552bed35-4de7-4ab3-8541-e4dee0ea1007 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:20.774082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-552bed35-4de7-4ab3-8541-e4dee0ea1007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:20.917336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1209/2420] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:20.918154 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:20.920159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:21.002026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63d04-8627-41d3-8cba-8d6e34599a15 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:21.014238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:21.015173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:21.015516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:21.015728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:21.028057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:21.029595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:21.051044 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1212/2421] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:22.201273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c29f32-0871-41fb-8cc5-cbd9838ae15a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:22.202125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c29f32-0871-41fb-8cc5-cbd9838ae15a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:22.202125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c29f32-0871-41fb-8cc5-cbd9838ae15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:22.278566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-84c29f32-0871-41fb-8cc5-cbd9838ae15a None None] MFA Rules not processed for user `d26c50b54171491b987ca844091c6e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:22.281829 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-84c29f32-0871-41fb-8cc5-cbd9838ae15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:22.392798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:22.491623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1210/2422] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:22.510331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:22.511271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:22.645761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f07d4-aa8c-4f14-81b4-01736d0a82a5 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:22.659349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:22.660107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:22.660476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:22.660754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:22.678499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:22.680259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:22.704462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1213/2423] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:24.207149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a55fc-01d1-4543-a25c-7a3954cb1107 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:24.207702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a55fc-01d1-4543-a25c-7a3954cb1107 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:24.207702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a55fc-01d1-4543-a25c-7a3954cb1107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:24.227289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-264a55fc-01d1-4543-a25c-7a3954cb1107 None None] MFA Rules not processed for user `d26c50b54171491b987ca844091c6e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:24.228720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-264a55fc-01d1-4543-a25c-7a3954cb1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:24.236716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1211/2424] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:32.698955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:34.222724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=13 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:35.775519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:35.777095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:35.796618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8869cdc-4959-4e71-8b38-d026a356349c service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:35.804933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:35.805479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:35.805590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:35.807990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:35.809274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:35.809779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:35.810376 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] PATH_INFO: `/v3/users/83f6c2bec232400da2cf79be12a16182` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:35.811993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f6c2bec232400da2cf79be12a16182)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:35.814252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:35.840284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:35.842229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:35.914003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f6c2bec232400da2cf79be12a16182'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:35.931098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f6c2bec232400da2cf79be12a16182'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:35.932221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f6c2bec232400da2cf79be12a16182'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:35.933235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f6c2bec232400da2cf79be12a16182'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:35.956681 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1212/2425] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:33:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:35.969023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6a39f3c4-817b-4ec1-aeab-392865d96157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f6c2bec232400da2cf79be12a16182'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:35.971817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:35.975904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:35.997568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1214/2426] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:35 2022] DELETE /identity/v3/users/83f6c2bec232400da2cf79be12a16182 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:36.015623 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:36.015913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:36.016200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:36.050626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:36.051884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:36.094079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e79461-13a5-496d-bb90-1983f2c986d8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:36.167797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1215/2427] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:36.184882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:36.189149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:36.222910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57571553-ee79-4dd0-b1f7-efef303799e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:36.223816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57571553-ee79-4dd0-b1f7-efef303799e1 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:36.225581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57571553-ee79-4dd0-b1f7-efef303799e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:36.226016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57571553-ee79-4dd0-b1f7-efef303799e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:36.347311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:36.366648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb515c-7c91-49e5-bd59-d18f92e34833 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:36.382353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:36.383082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:36.383483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:36.383758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:36.402321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:36.404409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57571553-ee79-4dd0-b1f7-efef303799e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:36.405074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:36.406338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57571553-ee79-4dd0-b1f7-efef303799e1 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:36.426429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1216/2428] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:36.439474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1213/2429] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 13:33:37.079744 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:37.081237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:37.106877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0a821-9dab-4136-9c90-ca06a86fd9f7 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:37.112990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:37.114326 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.114873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.115398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] PATH_INFO: `/v3/projects/3ac35d497508475599b3843fca19c15e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.117078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac35d497508475599b3843fca19c15e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:37.119576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:37.135132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:37.209358 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac35d497508475599b3843fca19c15e', 'request_id': 'req-8a417c86-8ba5-4224-9cca-1d134e496454'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:37.233397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac35d497508475599b3843fca19c15e', 'request_id': 'req-8a417c86-8ba5-4224-9cca-1d134e496454'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:37.240693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac35d497508475599b3843fca19c15e', 'request_id': 'req-8a417c86-8ba5-4224-9cca-1d134e496454'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:37.277214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] The token cache is being invalidate because project 3ac35d497508475599b3843fca19c15e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:37.277777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:37.278594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8a417c86-8ba5-4224-9cca-1d134e496454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:37.283276 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1217/2430] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:37 2022] DELETE /identity/v3/projects/3ac35d497508475599b3843fca19c15e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:37.288012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.288572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.289839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.347625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:37.350986 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:37.472771 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1214/2431] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:37.481285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.482189 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.482720 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.514083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:37.515668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:37.541320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1218/2432] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:37.544815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:37.561569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35946759-f7ae-45db-9584-97dcab4b3ee8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:37.566832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:37.567758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.568203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.568987 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.569969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:37.570456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:37.572865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:37.603114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1215/2433] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:33:37.604667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:37.622191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f7f5c-e173-43d7-97a9-7083d448d8ac None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:37.647196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:37.647911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.648215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.648476 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.649868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:37.651835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:37.749760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1219/2434] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:37 2022] POST /identity/v3/projects => generated 361 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:37.750583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:37.771296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5531fa-e7a3-4127-8ac1-cb1389cd5114 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:37.882566 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:37.883955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.884543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.884895 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.886065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:37.888128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:37.929464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1216/2435] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:37 2022] POST /identity/v3/users => generated 281 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:37.933114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:37.953939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dfb4e-a630-4393-b776-a41a91ed75fd admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:37.961277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:37.962283 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:37.963204 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:37.963539 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:37.964428 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:37.972333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:37.999022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1220/2436] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:37.999908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:38.014013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5f114-68d7-4b9c-ae94-57fbf25c7d2a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:38.020325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc78c7c-707d-4f86-a8a3-4dfe9305d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:38.021498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc78c7c-707d-4f86-a8a3-4dfe9305d251 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:38.022005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc78c7c-707d-4f86-a8a3-4dfe9305d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:38.022491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc78c7c-707d-4f86-a8a3-4dfe9305d251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:38.023445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc78c7c-707d-4f86-a8a3-4dfe9305d251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:38.025476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc78c7c-707d-4f86-a8a3-4dfe9305d251 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:38.039745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1217/2437] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:38.044806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:38.059848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197d9f-d857-479c-952a-644fc7dd2c6d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:38.065318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:38.066328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:38.066613 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:38.066892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] PATH_INFO: `/v3/projects/251e8daa68c74ed6be0026faeb98e163/users/7481de07f0cd4160beab384367bebae1/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:38.095314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] RBAC: Authorizing `identity:create_grant(project_id=251e8daa68c74ed6be0026faeb98e163, user_id=7481de07f0cd4160beab384367bebae1, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:38.098947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:38.135272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1221/2438] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:38 2022] PUT /identity/v3/projects/251e8daa68c74ed6be0026faeb98e163/users/7481de07f0cd4160beab384367bebae1/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:38.146498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:38.147417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:38.147765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:38.182954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] MFA Rules not processed for user `7481de07f0cd4160beab384367bebae1`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:38.184819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:38.229100 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:38.330076 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1218/2439] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:38.367069 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:38.369910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:38.476712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d2f7d-a373-4357-add8-67400551406d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:38.481700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef28702a-ee6c-4abc-8727-3c2d88c49ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:38.482629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28702a-ee6c-4abc-8727-3c2d88c49ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:38.483091 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28702a-ee6c-4abc-8727-3c2d88c49ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:38.483573 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef28702a-ee6c-4abc-8727-3c2d88c49ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:38.493980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28702a-ee6c-4abc-8727-3c2d88c49ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:38.495878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef28702a-ee6c-4abc-8727-3c2d88c49ee1 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:38.507649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1222/2440] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:38.525710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:38.662667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231d318-26f8-46b7-ac9b-bcc8bf494026 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:38.675911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:38.676832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:38.677779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:38.678251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] PATH_INFO: `/v3/users/7481de07f0cd4160beab384367bebae1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:38.679877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7481de07f0cd4160beab384367bebae1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:38.682802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:38.826795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7481de07f0cd4160beab384367bebae1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:38.863001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7481de07f0cd4160beab384367bebae1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:38.864581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7481de07f0cd4160beab384367bebae1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:38.865957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7481de07f0cd4160beab384367bebae1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:38.882152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7481de07f0cd4160beab384367bebae1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:38.909203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1219/2441] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:38 2022] DELETE /identity/v3/users/7481de07f0cd4160beab384367bebae1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:38.912965 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:38.913491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:38.913884 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:38.969225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:38.970594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:39.055507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:39.127479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1223/2442] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:39.161513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:39.163474 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:39.249277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:39.277297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711d155-02fc-407e-a549-b3b8c16764b6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:39.299415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad85a8-1e9d-4584-ba7c-8dd49c35862e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:39.303395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad85a8-1e9d-4584-ba7c-8dd49c35862e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:39.303875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad85a8-1e9d-4584-ba7c-8dd49c35862e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:39.304261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad85a8-1e9d-4584-ba7c-8dd49c35862e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:39.329402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad85a8-1e9d-4584-ba7c-8dd49c35862e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:39.331395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad85a8-1e9d-4584-ba7c-8dd49c35862e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:39.346592 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1220/2443] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:39.928165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:39.931003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:39.967860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270bd7e-b3ac-4639-8afc-9c79bc7d7fbf None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:39.981452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:39.982332 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:39.983284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:39.983718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] PATH_INFO: `/v3/projects/251e8daa68c74ed6be0026faeb98e163` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:39.985780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=251e8daa68c74ed6be0026faeb98e163)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:39.988164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:40.027300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:40.027803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:40.028225 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:40.051819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] MFA Rules not processed for user `2d3b91bfd0304cc793e3ad52c9b9e865`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:40.053511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:40.101643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251e8daa68c74ed6be0026faeb98e163', 'request_id': 'req-a0902332-5db1-4979-a258-404c5e499fb8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:40.117440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251e8daa68c74ed6be0026faeb98e163', 'request_id': 'req-a0902332-5db1-4979-a258-404c5e499fb8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:40.128802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251e8daa68c74ed6be0026faeb98e163', 'request_id': 'req-a0902332-5db1-4979-a258-404c5e499fb8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:40.144054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] The token cache is being invalidate because project 251e8daa68c74ed6be0026faeb98e163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:40.144710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:40.146210 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a0902332-5db1-4979-a258-404c5e499fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:40.172990 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1221/2445] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:40.179988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1224/2445] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:39 2022] DELETE /identity/v3/projects/251e8daa68c74ed6be0026faeb98e163 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:50.138218 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:50.140424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:51.005029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:51.006505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:51.020622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364edd4-6efd-4825-b0fc-6df3f67e327e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:51.088301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:51.089183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:51.089528 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:51.089806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] PATH_INFO: `/v3/users/ad09c35954de4a4a98bcdd9741dd1c37` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:51.091107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad09c35954de4a4a98bcdd9741dd1c37)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:51.093077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:51.149543 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad09c35954de4a4a98bcdd9741dd1c37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.165706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad09c35954de4a4a98bcdd9741dd1c37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.166433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad09c35954de4a4a98bcdd9741dd1c37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.167065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad09c35954de4a4a98bcdd9741dd1c37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.174468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad09c35954de4a4a98bcdd9741dd1c37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.192492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1222/2446] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:33:51 2022] DELETE /identity/v3/users/ad09c35954de4a4a98bcdd9741dd1c37 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:51.198124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:51.198690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:51.198690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:51.218081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:51.219491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:51.283562 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1225/2447] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:33:51.294165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:51.295793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:51.355720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78735c79-a37f-49fd-8b45-96426c55c899 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:51.359924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3b61f-d11a-47e7-8ece-740888c68e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:51.360583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3b61f-d11a-47e7-8ece-740888c68e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:51.360870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3b61f-d11a-47e7-8ece-740888c68e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:51.361145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3b61f-d11a-47e7-8ece-740888c68e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:51.371281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3b61f-d11a-47e7-8ece-740888c68e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:51.372971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3b61f-d11a-47e7-8ece-740888c68e96 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:51.382936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1223/2448] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:33:51.745108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:51.746672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:51.759629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220142-5ffb-45d8-ab6c-cdf4452b2e5c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:51.765348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:51.766690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:51.767136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:51.767808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] PATH_INFO: `/v3/projects/c27c3a7487164a079d01a93d2e7387a4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:51.775168 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27c3a7487164a079d01a93d2e7387a4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:51.778554 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:51.831661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27c3a7487164a079d01a93d2e7387a4', 'request_id': 'req-a2288513-c585-4f14-b5dc-b7257db03f36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.839895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27c3a7487164a079d01a93d2e7387a4', 'request_id': 'req-a2288513-c585-4f14-b5dc-b7257db03f36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.846089 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27c3a7487164a079d01a93d2e7387a4', 'request_id': 'req-a2288513-c585-4f14-b5dc-b7257db03f36'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.857120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] The token cache is being invalidate because project c27c3a7487164a079d01a93d2e7387a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:33:51.857425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.858093 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a2288513-c585-4f14-b5dc-b7257db03f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:33:51.859639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1226/2449] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:33:51 2022] DELETE /identity/v3/projects/c27c3a7487164a079d01a93d2e7387a4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:51.865264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:51.865501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:51.865811 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:51.884386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:51.885625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:51.909234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:51.958707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1224/2450] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:51.966291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:51.966586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:51.966867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:51.988524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:51.989755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:51.998117 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1227/2451] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:52.002385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.012767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b715be4-51c3-4447-b29b-c4f966f924a7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.017540 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.018205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.018489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.018764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.019525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:33:52.019832 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.021461 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.037823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1225/2452] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:33:52.038950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.049666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d050d-e39f-44db-a271-4f5173cbbee0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.055239 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.056384 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.056749 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.057170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.058173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.060232 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.086813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1228/2453] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:33:52 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:52.091285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.101639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdaf71-3136-41b7-91b1-cbf063718a04 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.167121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.168299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.168586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.168908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.169679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.171627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.193340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1226/2454] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:33:52 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:33:52.197650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.208136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0b23c-c056-4e8f-b790-2b0862731100 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.212776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.213462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.213743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.214016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.214752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.216446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.226739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1229/2455] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:33:52.230906 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.241177 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d58d97-5071-41a6-afcb-17e38172910c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.246024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91935ec0-7199-4590-bd76-43a67df3f697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.246808 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91935ec0-7199-4590-bd76-43a67df3f697 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.247096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91935ec0-7199-4590-bd76-43a67df3f697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.247412 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91935ec0-7199-4590-bd76-43a67df3f697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.248146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91935ec0-7199-4590-bd76-43a67df3f697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.249799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91935ec0-7199-4590-bd76-43a67df3f697 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.261651 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1227/2456] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:33:52.266165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.276440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0330125-ff05-48b6-98d1-57f88ec9c814 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.282695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.283655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.283937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.284244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] PATH_INFO: `/v3/projects/1755875da2df48a08e680353eb8937c1/users/3d640624e4c848a98f6d55ea88ffd109/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.300907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1755875da2df48a08e680353eb8937c1, user_id=3d640624e4c848a98f6d55ea88ffd109, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.303852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.323491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1230/2457] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:33:52 2022] PUT /identity/v3/projects/1755875da2df48a08e680353eb8937c1/users/3d640624e4c848a98f6d55ea88ffd109/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:33:52.340857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dd1d6-a939-4e00-90ee-3c5e06801480 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.341150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dd1d6-a939-4e00-90ee-3c5e06801480 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.341454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dd1d6-a939-4e00-90ee-3c5e06801480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.371261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-024dd1d6-a939-4e00-90ee-3c5e06801480 None None] MFA Rules not processed for user `3d640624e4c848a98f6d55ea88ffd109`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:52.372889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-024dd1d6-a939-4e00-90ee-3c5e06801480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:52.446392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1228/2458] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:33:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:52.458271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:52.460049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:52.485284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:52.548233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae2b1e-71e0-43e1-aac0-9546408b779b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:52.552830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:52.553805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:52.554255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:52.554542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:52.564635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:52.566147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:52.576037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1231/2459] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:55.279390 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:55.279924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:55.279924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:55.303512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:55.303512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:55.316514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:55.390247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1229/2460] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:33:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:33:55.406377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:55.408269 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:55.511214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f40089-1210-4eb6-b32a-ac1fcd4aab0b service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:55.516712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5158d-d001-4036-8818-035386ab3d5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:55.517367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5158d-d001-4036-8818-035386ab3d5f service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:55.517680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5158d-d001-4036-8818-035386ab3d5f service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:55.517994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5158d-d001-4036-8818-035386ab3d5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:55.532447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5158d-d001-4036-8818-035386ab3d5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:55.534008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5158d-d001-4036-8818-035386ab3d5f service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:55.555859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1232/2461] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:33:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:33:56.394752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:33:56.396259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:56.461838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52a538-72ac-43c8-b871-0d1de98f8aa2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:33:56.466103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:33:56.466806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:56.467193 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:56.467471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:56.470513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:56.542036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:33:56.543781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55212d-4f5c-4f30-a7e9-5d52ef0233db service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:33:56.555768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1230/2462] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:33:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:33:59.470311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5688e-37a8-4e73-b186-a9940e8e09cc None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:59.470876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5688e-37a8-4e73-b186-a9940e8e09cc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:59.470876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5688e-37a8-4e73-b186-a9940e8e09cc None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:59.472598 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1233/2463] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:33:59.499012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be508e-c303-4c9e-86f2-086614197583 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:33:59.500105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be508e-c303-4c9e-86f2-086614197583 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:33:59.500533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be508e-c303-4c9e-86f2-086614197583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:33:59.619615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d3be508e-c303-4c9e-86f2-086614197583 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:33:59.621162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d3be508e-c303-4c9e-86f2-086614197583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:33:59.717016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:33:59.861780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1231/2464] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:33:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:34:05.155675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5688e-37a8-4e73-b186-a9940e8e09cc None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:05.158208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b0e5688e-37a8-4e73-b186-a9940e8e09cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:05.261434 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5688e-37a8-4e73-b186-a9940e8e09cc None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:05.266324 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a56e0-866c-4a45-bcbf-f82503f82b7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:05.266992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a56e0-866c-4a45-bcbf-f82503f82b7a service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.267387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a56e0-866c-4a45-bcbf-f82503f82b7a service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.267672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a56e0-866c-4a45-bcbf-f82503f82b7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.286979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a56e0-866c-4a45-bcbf-f82503f82b7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:05.286979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a56e0-866c-4a45-bcbf-f82503f82b7a service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:05.294675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1234/2465] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:34:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:34:05.371864 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda409bf-cf66-4db8-8793-569615506429 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.372597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda409bf-cf66-4db8-8793-569615506429 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.372597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda409bf-cf66-4db8-8793-569615506429 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.374165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1232/2466] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:34:05.387291 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a91579-b243-40db-8b37-8e34e601b010 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.387583 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a91579-b243-40db-8b37-8e34e601b010 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.387962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a91579-b243-40db-8b37-8e34e601b010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.412077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-f0a91579-b243-40db-8b37-8e34e601b010 None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:05.413605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f0a91579-b243-40db-8b37-8e34e601b010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:05.422343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1235/2467] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:34:05.436590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda409bf-cf66-4db8-8793-569615506429 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:05.450797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda409bf-cf66-4db8-8793-569615506429 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:05.456950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabea80-148c-4749-b502-7c747887a12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:05.458029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabea80-148c-4749-b502-7c747887a12d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.458476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabea80-148c-4749-b502-7c747887a12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.459241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabea80-148c-4749-b502-7c747887a12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.469037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabea80-148c-4749-b502-7c747887a12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:05.470560 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabea80-148c-4749-b502-7c747887a12d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:05.481513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1233/2468] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:34:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:34:05.570314 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdefbae-6272-4798-91b0-f27bbc9300f1 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.570704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdefbae-6272-4798-91b0-f27bbc9300f1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.571074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdefbae-6272-4798-91b0-f27bbc9300f1 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.572607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1236/2469] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:34:05.579966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.580262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.580696 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.599745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:05.600958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:05.611991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1234/2470] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:34:05.622983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdefbae-6272-4798-91b0-f27bbc9300f1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:05.635032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdefbae-6272-4798-91b0-f27bbc9300f1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:05.641722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941364b-72df-43b3-9bde-bdbd2d1db38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:05.642395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941364b-72df-43b3-9bde-bdbd2d1db38e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:05.642674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941364b-72df-43b3-9bde-bdbd2d1db38e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:05.642947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941364b-72df-43b3-9bde-bdbd2d1db38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:05.654372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:05.657280 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941364b-72df-43b3-9bde-bdbd2d1db38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:05.659343 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941364b-72df-43b3-9bde-bdbd2d1db38e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:05.671728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1237/2471] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:34:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:34:06.559223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:06.626517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fe9b1-2bf1-44b6-8c61-e9d07c26b178 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:06.631452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:06.632347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:06.632626 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:06.632899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] PATH_INFO: `/v3/users/3d640624e4c848a98f6d55ea88ffd109` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:06.634147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d640624e4c848a98f6d55ea88ffd109)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:06.636134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:06.688111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d640624e4c848a98f6d55ea88ffd109'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:06.700087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d640624e4c848a98f6d55ea88ffd109'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:06.700748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d640624e4c848a98f6d55ea88ffd109'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:06.701357 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d640624e4c848a98f6d55ea88ffd109'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:06.708335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d640624e4c848a98f6d55ea88ffd109'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:06.725186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1235/2472] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:34:06 2022] DELETE /identity/v3/users/3d640624e4c848a98f6d55ea88ffd109 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:34:06.730057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:06.730625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:06.730625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:06.749462 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:06.750736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:06.810307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1238/2473] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:34:06.823810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:06.826364 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:06.864526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:06.901727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c430c4-9197-4754-b574-77eb45321478 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:06.905741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed59a9-6644-453a-8032-9251ef8e1ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:06.906393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed59a9-6644-453a-8032-9251ef8e1ace service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:06.906700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed59a9-6644-453a-8032-9251ef8e1ace service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:06.906974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed59a9-6644-453a-8032-9251ef8e1ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:06.916544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed59a9-6644-453a-8032-9251ef8e1ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:06.918639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed59a9-6644-453a-8032-9251ef8e1ace service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:06.928727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1236/2474] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:34:07.241148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:07.242778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:07.257147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ee55-4f66-4d16-809b-ff78bc3e4049 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:07.267511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:07.268630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:07.268916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:07.272458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] PATH_INFO: `/v3/projects/1755875da2df48a08e680353eb8937c1` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:07.280177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1755875da2df48a08e680353eb8937c1)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:07.282473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:07.335134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1755875da2df48a08e680353eb8937c1', 'request_id': 'req-879681d7-5357-4c67-8764-83d0da88030e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:07.344104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1755875da2df48a08e680353eb8937c1', 'request_id': 'req-879681d7-5357-4c67-8764-83d0da88030e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:07.350033 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1755875da2df48a08e680353eb8937c1', 'request_id': 'req-879681d7-5357-4c67-8764-83d0da88030e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:07.362668 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] The token cache is being invalidate because project 1755875da2df48a08e680353eb8937c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:34:07.362954 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:07.363697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:07.365227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1239/2475] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:34:07 2022] DELETE /identity/v3/projects/1755875da2df48a08e680353eb8937c1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:34:09.267487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880833cf-f2d7-40b9-9cd8-c5520c3200b2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:09.267957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880833cf-f2d7-40b9-9cd8-c5520c3200b2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:09.267957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880833cf-f2d7-40b9-9cd8-c5520c3200b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:09.322365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-880833cf-f2d7-40b9-9cd8-c5520c3200b2 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:09.323778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-880833cf-f2d7-40b9-9cd8-c5520c3200b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:09.407867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1237/2476] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:34:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:34:09.424284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:09.425669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:09.523776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:09.531902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879681d7-5357-4c67-8764-83d0da88030e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:09.539190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:09.539930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:09.540278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:09.540616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:09.550927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:09.552697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:09.563967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1240/2477] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:34:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:34:19.099731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:19.100455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:19.100455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:19.119699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:19.121103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:19.130294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1238/2478] 10.222.0.24 () {58 vars in 967 bytes} [Thu May 19 13:34:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:34:19.143042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:19.155077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b906c0-4721-4ebe-805f-9778427cef93 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:19.160078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:19.161835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:19.161835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:19.161835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:19.171537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:19.173625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:19.183247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1241/2479] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:34:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:34:21.333772 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:21.335532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:21.407890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1735629-2e81-4e74-8149-dd27107e53e2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:21.412831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:21.413752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:21.414036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:21.414339 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] PATH_INFO: `/v3/users/d9a8a480afb3438cb269ebd710c78018` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:21.415585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a8a480afb3438cb269ebd710c78018)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:21.418037 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:21.461319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:21.472121 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a8a480afb3438cb269ebd710c78018'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:21.486039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a8a480afb3438cb269ebd710c78018'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:21.486764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a8a480afb3438cb269ebd710c78018'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:21.487375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a8a480afb3438cb269ebd710c78018'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:21.494716 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9f03220c-2eb7-4f39-85be-f1f801d00269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a8a480afb3438cb269ebd710c78018'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:21.510955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1239/2480] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:34:21 2022] DELETE /identity/v3/users/d9a8a480afb3438cb269ebd710c78018 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:34:21.852883 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:21.865264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c37aa8-820e-4e6b-a0fc-17ad5200e907 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:21.926492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:21.927364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:21.927643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:21.927920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] PATH_INFO: `/v3/projects/5a26987661f54291b3cf7086b0a1c98c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:21.929617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a26987661f54291b3cf7086b0a1c98c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:21.931241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:21.984788 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a26987661f54291b3cf7086b0a1c98c', 'request_id': 'req-19610c05-7788-49ca-a489-6c080cb29a86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:21.993547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a26987661f54291b3cf7086b0a1c98c', 'request_id': 'req-19610c05-7788-49ca-a489-6c080cb29a86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:22.001502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a26987661f54291b3cf7086b0a1c98c', 'request_id': 'req-19610c05-7788-49ca-a489-6c080cb29a86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:22.015132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] The token cache is being invalidate because project 5a26987661f54291b3cf7086b0a1c98c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:34:22.015425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:22.016166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:34:22.017857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1242/2481] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:34:21 2022] DELETE /identity/v3/projects/5a26987661f54291b3cf7086b0a1c98c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:34:31.507283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:32.011523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:41.839984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:41.840504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:41.840504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:41.862270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:41.863533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:41.928892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1240/2482] 10.222.0.24 () {58 vars in 967 bytes} [Thu May 19 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:34:41.947042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:41.948571 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:42.019054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610c05-7788-49ca-a489-6c080cb29a86 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:42.024057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0e58d-ba04-410b-a34b-c1a3586172e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:42.024715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0e58d-ba04-410b-a34b-c1a3586172e0 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:42.025001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0e58d-ba04-410b-a34b-c1a3586172e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:42.025281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0e58d-ba04-410b-a34b-c1a3586172e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:42.033401 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0e58d-ba04-410b-a34b-c1a3586172e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:42.035023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0e58d-ba04-410b-a34b-c1a3586172e0 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:42.048016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1243/2483] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:34:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:34:50.407280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:34:50.408817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:50.518493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85406e3-ad50-4730-a5f2-f51c2786edbe None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:34:50.526437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:34:50.527157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:50.527417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:50.527725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:50.530806 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:50.542715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:34:50.544624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a40bd2-44d1-46b2-a2fb-6eddaee7f18c service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:34:50.554852 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1241/2484] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:34:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:34:52.039813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=32 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:34:59.806915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78244627-fe49-41df-bdd9-6d4dc705b848 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:34:59.807559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78244627-fe49-41df-bdd9-6d4dc705b848 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:34:59.807559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78244627-fe49-41df-bdd9-6d4dc705b848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:34:59.826791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-78244627-fe49-41df-bdd9-6d4dc705b848 None None] MFA Rules not processed for user `d26c50b54171491b987ca844091c6e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:34:59.828474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-78244627-fe49-41df-bdd9-6d4dc705b848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:34:59.885669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1244/2485] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:34:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:35:00.550619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=66 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:35:09.877737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:35:36.778264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83535-fde7-4f61-9ded-cb80bc78fbb2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:35:36.778860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83535-fde7-4f61-9ded-cb80bc78fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:35:36.778860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f83535-fde7-4f61-9ded-cb80bc78fbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:35:36.798920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-80f83535-fde7-4f61-9ded-cb80bc78fbb2 None None] MFA Rules not processed for user `14da69798e344e70ba64bf8187523e45`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:35:36.800421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-80f83535-fde7-4f61-9ded-cb80bc78fbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:35:36.864569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1242/2486] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:35:37.947608 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:35:37.948147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:35:37.948305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:35:37.969636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] MFA Rules not processed for user `2d3b91bfd0304cc793e3ad52c9b9e865`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:35:37.970857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:35:38.028855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1245/2487] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:35:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:35:46.855962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=25 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:35:48.020414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:35:59.777433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:35:59.777913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:35:59.778015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:35:59.797310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:35:59.798595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:35:59.808708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1243/2488] 10.222.0.24 () {58 vars in 967 bytes} [Thu May 19 13:35:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:35:59.860776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:35:59.873077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46859e0a-17f9-4955-ab82-22971d5045fb None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:35:59.878892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:35:59.879560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:35:59.879848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:35:59.880119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:35:59.889336 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:35:59.891070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:35:59.900834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1246/2489] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:36:01.497621 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:01.499955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:01.584919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96578ee-f877-4efe-9194-df04bf5feaa9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:01.591384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:01.595279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:01.595658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:01.596030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] PATH_INFO: `/v3/users/d26c50b54171491b987ca844091c6e73` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:01.597934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26c50b54171491b987ca844091c6e73)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:01.600817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:01.672674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26c50b54171491b987ca844091c6e73'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:01.687069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26c50b54171491b987ca844091c6e73'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:01.687820 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26c50b54171491b987ca844091c6e73'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:01.688448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26c50b54171491b987ca844091c6e73'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:01.696713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1991060-1548-4ba9-8281-b0a72c7aa16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26c50b54171491b987ca844091c6e73'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:01.712044 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1244/2490] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:36:01 2022] DELETE /identity/v3/users/d26c50b54171491b987ca844091c6e73 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:02.042266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.054829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e737e9-3e25-453a-b0c9-06d8eea85530 service swift] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.121132 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.122219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.122569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.123652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] PATH_INFO: `/v3/projects/c96a605108ec47a081c60bc8dc16fd8e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.125333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96a605108ec47a081c60bc8dc16fd8e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.127957 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.189087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96a605108ec47a081c60bc8dc16fd8e', 'request_id': 'req-435b0379-6ffd-41dc-838a-5e1219f6014e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:02.198350 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96a605108ec47a081c60bc8dc16fd8e', 'request_id': 'req-435b0379-6ffd-41dc-838a-5e1219f6014e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:02.206860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96a605108ec47a081c60bc8dc16fd8e', 'request_id': 'req-435b0379-6ffd-41dc-838a-5e1219f6014e'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:02.222411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] The token cache is being invalidate because project c96a605108ec47a081c60bc8dc16fd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:36:02.222802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:02.223597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-435b0379-6ffd-41dc-838a-5e1219f6014e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:02.225252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1247/2491] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:36:02 2022] DELETE /identity/v3/projects/c96a605108ec47a081c60bc8dc16fd8e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:02.230371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.230936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.231520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.256169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:02.257391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:02.330106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1245/2492] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:36:02.338383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.338592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.338750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.359881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:02.361267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:02.370927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1248/2493] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:02.375022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.386831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0082-f41e-4366-8c65-9fc9e0d73647 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.393804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.394706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.395066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.395348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.396191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:36:02.396584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.398337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.409790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:02.414152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1246/2494] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:36:02.418865 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.429635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec9eba-a5b7-4403-bf66-40494e3d734f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.435024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.435730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.436015 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.436294 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.437000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.439355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.470309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1249/2495] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:36:02 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:36:02.474424 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.486994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f322d-3a36-451f-bde8-77c6353297cb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.548736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.549569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.549837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.550190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.550973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.553453 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.581368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1247/2496] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:36:02 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:36:02.582801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.596255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a2454-b954-400f-9fb4-c2ad361470d3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.601271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.602098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.602452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.602780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.603602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.605317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.619055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1250/2497] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:02.623088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.635795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691601f-358c-4a63-b782-00366b265294 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.642334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7ce4b-a469-4e04-add5-02962a17a82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.642973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ce4b-a469-4e04-add5-02962a17a82d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.643222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ce4b-a469-4e04-add5-02962a17a82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.643689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7ce4b-a469-4e04-add5-02962a17a82d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.644676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7ce4b-a469-4e04-add5-02962a17a82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.646478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7ce4b-a469-4e04-add5-02962a17a82d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.658283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1248/2498] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:02.662822 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.678013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe161bc-61e6-46b8-b862-c3f37380104d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.683518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.684544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.684895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.685240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] PATH_INFO: `/v3/projects/d60f500e123d4c73b38b952329d4d188/users/a95694dd815841abbaaebca3ef385fa7/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.704982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60f500e123d4c73b38b952329d4d188, user_id=a95694dd815841abbaaebca3ef385fa7, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.710787 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.738223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1251/2499] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:36:02 2022] PUT /identity/v3/projects/d60f500e123d4c73b38b952329d4d188/users/a95694dd815841abbaaebca3ef385fa7/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:36:02.749360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.749862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.750265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.780348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] MFA Rules not processed for user `a95694dd815841abbaaebca3ef385fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:02.781627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:02.856821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1249/2500] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:36:02.868209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:02.870284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:02.886614 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:02.941821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa084e8-db41-4e7a-82bc-10ebd5a77614 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:02.945559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:02.946215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:02.946498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:02.946777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:02.955885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:02.957333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:02.967085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1252/2501] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:07.272469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:07.274195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:07.348128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7e53f-3072-454f-a182-710418ad5a14 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:07.353914 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:07.354805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:07.355092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:07.355374 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] PATH_INFO: `/v3/users/b11df35c0e4541deaea0494759301a94` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:07.356654 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11df35c0e4541deaea0494759301a94)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:07.358680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:07.370131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:07.411223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11df35c0e4541deaea0494759301a94'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.423878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11df35c0e4541deaea0494759301a94'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.424603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11df35c0e4541deaea0494759301a94'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.425211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11df35c0e4541deaea0494759301a94'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.432440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11df35c0e4541deaea0494759301a94'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.447901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1250/2502] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:36:07 2022] DELETE /identity/v3/users/b11df35c0e4541deaea0494759301a94 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:07.642811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:07.660012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eccd45-dba2-4262-a31b-eac9d5dc97fd service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:07.732248 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:07.733191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:07.733519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:07.733979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] PATH_INFO: `/v3/projects/510387333865470196f9be30ef62395c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:07.735449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510387333865470196f9be30ef62395c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:07.739885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:07.794109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510387333865470196f9be30ef62395c', 'request_id': 'req-f8b86a82-4d1c-43d2-9c49-494569a99878'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.805933 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510387333865470196f9be30ef62395c', 'request_id': 'req-f8b86a82-4d1c-43d2-9c49-494569a99878'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.812662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510387333865470196f9be30ef62395c', 'request_id': 'req-f8b86a82-4d1c-43d2-9c49-494569a99878'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.824072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] The token cache is being invalidate because project 510387333865470196f9be30ef62395c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:36:07.824421 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.825112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.826671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1253/2503] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:36:07 2022] DELETE /identity/v3/projects/510387333865470196f9be30ef62395c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:07.830924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:07.832945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:07.846700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a80dc5-5dbc-4d88-ade9-e9c9ea0b7253 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:07.910564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:07.911443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:07.911845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:07.912060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] PATH_INFO: `/v3/users/9236c3e6183145d0a873b7c59754087b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:07.913247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9236c3e6183145d0a873b7c59754087b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:07.915206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:07.964694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9236c3e6183145d0a873b7c59754087b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.977378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9236c3e6183145d0a873b7c59754087b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.978341 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9236c3e6183145d0a873b7c59754087b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.978982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9236c3e6183145d0a873b7c59754087b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.986812 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9236c3e6183145d0a873b7c59754087b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:07.999671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1251/2504] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:36:07 2022] DELETE /identity/v3/users/9236c3e6183145d0a873b7c59754087b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:36:08.199601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:08.201882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:08.215802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b86a82-4d1c-43d2-9c49-494569a99878 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:08.262236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:08.277904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:08.278826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:08.279104 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:08.279379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] PATH_INFO: `/v3/projects/aa772f930d6a41bbab9ccfc2bd4d8fd0` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:08.280615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa772f930d6a41bbab9ccfc2bd4d8fd0)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:08.282727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:08.329643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa772f930d6a41bbab9ccfc2bd4d8fd0', 'request_id': 'req-6d358ed2-07d1-4902-ac02-46b1888650c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:08.337023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa772f930d6a41bbab9ccfc2bd4d8fd0', 'request_id': 'req-6d358ed2-07d1-4902-ac02-46b1888650c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:08.342511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa772f930d6a41bbab9ccfc2bd4d8fd0', 'request_id': 'req-6d358ed2-07d1-4902-ac02-46b1888650c8'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:08.354635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] The token cache is being invalidate because project aa772f930d6a41bbab9ccfc2bd4d8fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:36:08.354914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:08.355538 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-6d358ed2-07d1-4902-ac02-46b1888650c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:08.357123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1254/2505] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:36:08 2022] DELETE /identity/v3/projects/aa772f930d6a41bbab9ccfc2bd4d8fd0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:17.996184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=20,keystone:SELECT=75,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:18.351125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:19.318200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:19.319880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:19.335156 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4617-dbfb-4bcb-9b8a-c2f431ed9054 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:19.396865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:19.397781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:19.398067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:19.398347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] PATH_INFO: `/v3/users/a95694dd815841abbaaebca3ef385fa7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:19.399567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95694dd815841abbaaebca3ef385fa7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:19.401698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:19.453949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95694dd815841abbaaebca3ef385fa7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:19.466274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95694dd815841abbaaebca3ef385fa7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:19.466933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95694dd815841abbaaebca3ef385fa7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:19.467531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95694dd815841abbaaebca3ef385fa7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:19.475089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95694dd815841abbaaebca3ef385fa7'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:19.490662 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1252/2506] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:36:19 2022] DELETE /identity/v3/users/a95694dd815841abbaaebca3ef385fa7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:19.495472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:19.495986 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:19.496092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:19.514456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:19.515649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:19.576452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1255/2507] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:19.586597 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:19.588978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:19.648785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea804789-0c54-4177-8c17-89622b855775 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:19.653276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4e946-465e-48f5-a935-b770a68cacfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:19.653961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e946-465e-48f5-a935-b770a68cacfd service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:19.654247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e946-465e-48f5-a935-b770a68cacfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:19.654524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4e946-465e-48f5-a935-b770a68cacfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:19.664335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e946-465e-48f5-a935-b770a68cacfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:19.665817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4e946-465e-48f5-a935-b770a68cacfd service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:19.675819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1253/2508] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:19.960507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:19.962063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:19.975458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bed80-1ed9-458e-8f9a-53d324e31b43 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:19.980675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:19.981605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:19.981874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:19.982184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] PATH_INFO: `/v3/projects/d60f500e123d4c73b38b952329d4d188` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:19.983400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60f500e123d4c73b38b952329d4d188)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:19.985791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.038039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60f500e123d4c73b38b952329d4d188', 'request_id': 'req-1a03492e-9bc8-4b23-978f-252d1ad459d1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:20.047489 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60f500e123d4c73b38b952329d4d188', 'request_id': 'req-1a03492e-9bc8-4b23-978f-252d1ad459d1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:20.054191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60f500e123d4c73b38b952329d4d188', 'request_id': 'req-1a03492e-9bc8-4b23-978f-252d1ad459d1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:20.066410 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] The token cache is being invalidate because project d60f500e123d4c73b38b952329d4d188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:36:20.066705 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:20.067357 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-1a03492e-9bc8-4b23-978f-252d1ad459d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:20.068946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1256/2509] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:36:19 2022] DELETE /identity/v3/projects/d60f500e123d4c73b38b952329d4d188 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:20.074377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.074667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.074949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.093865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:20.095322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:20.114012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:20.160307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1254/2510] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:20.168444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.168730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.169346 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.189261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:20.190661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:20.199819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1257/2511] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:20.204400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.218771 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46573a4-c9b4-4eff-b1c1-ad3d46d50a38 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.224641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.225322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.225644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.225956 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.226689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:36:20.227006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.228699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.241495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1255/2512] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:36:20.246018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.255568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370ffa9-c6d0-43fb-890d-9becbd0f5a90 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.260488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.261145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.261440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.261718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.262431 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.264340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.292693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1258/2513] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:36:20 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:36:20.297314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.307418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2c710-39c9-4c27-9038-a9f5d23a3c3d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.366338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.367041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.367279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.367554 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.368249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.370209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.392073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1256/2514] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:36:20 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:36:20.396345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.406689 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357a785-c929-41b2-8e6f-4efb24f67501 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.411416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.412046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.412359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.412627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.413320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.414974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.426241 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1259/2515] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:20.430816 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.445966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418f7d1-a679-4b6d-bf91-4c632d80d66d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.454574 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd0799-c29d-4d96-a138-75e800088668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.455246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd0799-c29d-4d96-a138-75e800088668 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.455530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd0799-c29d-4d96-a138-75e800088668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.455807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd0799-c29d-4d96-a138-75e800088668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.456569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd0799-c29d-4d96-a138-75e800088668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.458242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd0799-c29d-4d96-a138-75e800088668 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.468646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1257/2516] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:20.472796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.484441 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7210750-593d-48b7-81ee-84f543066581 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.490123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.491019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.491309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.492252 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] PATH_INFO: `/v3/projects/98ad7b921dbc4a8d9e9be0d50385deee/users/aa0a4b41707d444581ee0aa2296ac0ba/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.508393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98ad7b921dbc4a8d9e9be0d50385deee, user_id=aa0a4b41707d444581ee0aa2296ac0ba, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.511124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.527695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1260/2517] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:36:20 2022] PUT /identity/v3/projects/98ad7b921dbc4a8d9e9be0d50385deee/users/aa0a4b41707d444581ee0aa2296ac0ba/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:20.532965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.533256 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.533584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.552335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:20.553516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:20.616830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1258/2518] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:20.628191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:20.630633 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:20.654445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:20.692620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf92096-5fc5-49c7-a44e-ffa20dd81433 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:20.696648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb06e2-5bfd-40aa-8f7f-c43da2df854f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:20.697290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb06e2-5bfd-40aa-8f7f-c43da2df854f service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:20.697605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb06e2-5bfd-40aa-8f7f-c43da2df854f service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:20.697878 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb06e2-5bfd-40aa-8f7f-c43da2df854f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:20.707369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb06e2-5bfd-40aa-8f7f-c43da2df854f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:20.708794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb06e2-5bfd-40aa-8f7f-c43da2df854f service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:20.718713 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1261/2519] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:24.187300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:24.190355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:24.257516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:24.310451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae101487-2385-4f0c-ac77-6ba62f9d1a16 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:24.315472 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:24.316182 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:24.316486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:24.316764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:24.318691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:24.424053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:24.425709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:24.453639 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1259/2520] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:25.213103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.213643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.213727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.259909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] MFA Rules not processed for user `aa0a4b41707d444581ee0aa2296ac0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:25.259909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:25.331235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1262/2521] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:25.335802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:25.347227 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f23b-559d-42a0-b798-3e7f6b74bff2 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:25.351783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:25.352596 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.352884 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.353161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.353913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:25.355865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:25.384855 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1260/2522] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:36:25 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:36:25.388533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:25.400009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258cb0-72e0-49cb-8f8c-7279e91493b6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:25.458650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:25.459340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.459644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.459971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.460666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:25.462716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:25.486874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1263/2523] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:36:25 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:36:25.491229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:25.502663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929566d-6456-4c33-8363-d44bcdafe94d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:25.508038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:25.508719 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.509002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.509278 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.510050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:25.511687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:25.523813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1261/2524] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:36:25.528199 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:25.539319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe134b-cd29-4e4d-9d2b-c3148ccf1cfe admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:25.545307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:25.548185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.548185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.548531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] PATH_INFO: `/v3/projects/1564d5d1d11f43fa9abc64a0fe12818a/users/e9ed9b0406de40df8843e3e137a1303d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.566215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1564d5d1d11f43fa9abc64a0fe12818a, user_id=e9ed9b0406de40df8843e3e137a1303d, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:25.569265 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:25.585537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1264/2525] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:36:25 2022] PUT /identity/v3/projects/1564d5d1d11f43fa9abc64a0fe12818a/users/e9ed9b0406de40df8843e3e137a1303d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:36:25.589370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:25.604299 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926821c9-661e-461b-91d4-2f2ff58740e4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:25.673902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:25.674763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.674998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.675363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.676238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:25.678239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:25.689981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1262/2526] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:25.694028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:25.706947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde567e-87ab-4c41-a019-d6173fc854bc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:25.711888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54913c-74fd-4adf-a382-2758947e0667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:25.712932 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54913c-74fd-4adf-a382-2758947e0667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:25.713310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54913c-74fd-4adf-a382-2758947e0667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:25.713712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54913c-74fd-4adf-a382-2758947e0667 admin admin] PATH_INFO: `/v3/projects/1564d5d1d11f43fa9abc64a0fe12818a/users/e9ed9b0406de40df8843e3e137a1303d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:25.715855 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54913c-74fd-4adf-a382-2758947e0667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1564d5d1d11f43fa9abc64a0fe12818a, user_id=e9ed9b0406de40df8843e3e137a1303d, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:25.718710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54913c-74fd-4adf-a382-2758947e0667 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:25.734763 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1265/2527] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:36:25 2022] PUT /identity/v3/projects/1564d5d1d11f43fa9abc64a0fe12818a/users/e9ed9b0406de40df8843e3e137a1303d/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:27.094730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:27.148581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:27.178802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fa140-e7b2-45e5-8f46-a27831c1c929 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:27.253253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:27.253905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:27.254190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:27.254464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:27.258912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:27.382552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:27.389905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:27.402729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1263/2528] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:30.242866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:30.243359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:30.243456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:30.250617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:30.276074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] MFA Rules not processed for user `e9ed9b0406de40df8843e3e137a1303d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:30.277478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:30.351827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1266/2529] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:30.366032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:30.367672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:30.381631 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d72e9-ab7b-4477-8281-f7c8b0c14df7 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:30.387484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:30.388132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:30.388418 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:30.388702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:30.454698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:30.456324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:30.467749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1264/2530] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:34.474394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:34.476216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:34.494235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d409bcc-98fc-48b2-8216-67e7e8de124a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:34.500414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:34.501617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:34.502002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:34.502284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:34.506448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:34.569558 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:34.571377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:34.582287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1267/2531] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:36.834680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:36.848402 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60fbc2-38a3-43da-8ebb-d3669cfdb3c4 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:36.855068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:36.855723 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:36.856008 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:36.856285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:36.858148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:36.888710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:36.927644 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:36.929272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93c695-bfbe-4852-a0b7-1acfea676154 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:36.938680 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1265/2532] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:44.576671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=60 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:45.416828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:45.434419 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ad432-9e67-4d3e-8a2c-10aebba4a6ff service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:45.440994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:45.441728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:45.442235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:45.442860 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:45.445625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:45.542096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:45.543618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:45.556447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1268/2533] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:46.934579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=15 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:48.507619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:48.508675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:48.509049 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:48.526792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] MFA Rules not processed for user `aa0a4b41707d444581ee0aa2296ac0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:48.528086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:48.535648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1266/2534] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:49.131402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:49.133022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:49.149271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfac6cc-d6a9-413c-8486-f265835fd6e4 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:49.154312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:49.155192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.155479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.155756 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] PATH_INFO: `/v3/users/2d3b91bfd0304cc793e3ad52c9b9e865` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.156979 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3b91bfd0304cc793e3ad52c9b9e865)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:49.159208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:49.210315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3b91bfd0304cc793e3ad52c9b9e865'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.219593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3b91bfd0304cc793e3ad52c9b9e865'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.220414 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3b91bfd0304cc793e3ad52c9b9e865'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.221087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3b91bfd0304cc793e3ad52c9b9e865'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.234034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3b91bfd0304cc793e3ad52c9b9e865'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.248342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1269/2535] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:36:49 2022] DELETE /identity/v3/users/2d3b91bfd0304cc793e3ad52c9b9e865 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:49.476437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:49.488065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f24d4d-4c11-4408-ab15-0c23d757c4c8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:49.575726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:49.576968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.577509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.578284 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] PATH_INFO: `/v3/projects/63972a518bb048d19dd070eb9ac85445` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.579828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63972a518bb048d19dd070eb9ac85445)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:49.582053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:49.609183 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:49.611563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:49.669469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63972a518bb048d19dd070eb9ac85445', 'request_id': 'req-19fa92bd-4741-486f-9d12-d3a9c316c25b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.679456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63972a518bb048d19dd070eb9ac85445', 'request_id': 'req-19fa92bd-4741-486f-9d12-d3a9c316c25b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.686789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63972a518bb048d19dd070eb9ac85445', 'request_id': 'req-19fa92bd-4741-486f-9d12-d3a9c316c25b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.704541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] The token cache is being invalidate because project 63972a518bb048d19dd070eb9ac85445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:36:49.705763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.705958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-19fa92bd-4741-486f-9d12-d3a9c316c25b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:36:49.708509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1267/2536] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:36:49 2022] DELETE /identity/v3/projects/63972a518bb048d19dd070eb9ac85445 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:49.714127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde8e27-992e-4e59-9f0c-a736cbed65dd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.714608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde8e27-992e-4e59-9f0c-a736cbed65dd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.714946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde8e27-992e-4e59-9f0c-a736cbed65dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.733958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a85497-3582-4694-986a-d9db3c71cb3a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:49.739511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fbde8e27-992e-4e59-9f0c-a736cbed65dd None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:49.740678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fbde8e27-992e-4e59-9f0c-a736cbed65dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:49.741137 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:49.807974 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:49.808634 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.808915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.809191 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.811046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:49.811973 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1268/2537] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:36:49.819391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.819686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.819962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.823995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:49.825675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:49.837580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1270/2538] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:36:49.838231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:49.839424 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:49.847458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1269/2539] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:36:49.851722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:49.862128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921b818-ed7b-4059-9c9a-052cd91d37d1 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:49.866818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:49.867472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.867777 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.868051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.868768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:36:49.869072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:49.870687 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:49.883180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1271/2540] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:36:49.888606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:49.898727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3da35-75aa-4e06-ba05-47944adb8f3f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:49.905038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:49.905760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:49.906181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:49.906466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:49.907169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:49.909466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:49.925649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:49.941679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1270/2541] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:36:49 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:36:49.946902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:49.957576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e6c7d-bbce-4f0d-9b21-ad766baac75e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:50.023315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:50.024055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:50.024368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:50.024586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:50.025303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:50.027308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:50.049450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1272/2542] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:36:49 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:36:50.053440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:50.063960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e13439-d1d6-4988-810c-29f93d5280a9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:50.068857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:50.070589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:50.070878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:50.071151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:50.071898 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:50.074122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:50.087925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1271/2543] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:50.092656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:50.103984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3373eb-509c-4efe-a5bb-15898dc9fdc8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:50.108964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc7381-9047-4147-bf63-60e87106b8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:50.109673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc7381-9047-4147-bf63-60e87106b8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:50.109948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc7381-9047-4147-bf63-60e87106b8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:50.110244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc7381-9047-4147-bf63-60e87106b8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:50.111021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc7381-9047-4147-bf63-60e87106b8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:50.112654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc7381-9047-4147-bf63-60e87106b8e9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:50.125024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1273/2544] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:36:50.129704 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:50.140408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da534-9c95-4eee-af86-1aca0a040bc5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:50.145689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:50.146591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:50.146873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:50.147151 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] PATH_INFO: `/v3/projects/b1222038740244f69a977fe658d9bfe0/users/6883da006d3f4aec8805c74cf5ffae86/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:50.164055 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1222038740244f69a977fe658d9bfe0, user_id=6883da006d3f4aec8805c74cf5ffae86, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:50.167338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:50.183865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1272/2545] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:36:50 2022] PUT /identity/v3/projects/b1222038740244f69a977fe658d9bfe0/users/6883da006d3f4aec8805c74cf5ffae86/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:36:50.189969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:50.190459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:50.190757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:50.212221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:50.213626 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:50.286068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1274/2546] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:50.297375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:50.299303 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:50.365516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134124-be7d-488f-910c-bf091c197ac5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:50.369607 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:50.370267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:50.370549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:50.370823 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:50.380730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:50.382222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:50.392027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1273/2547] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:36:51.831646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:51.839068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:51.847413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee751a-bdc4-4c8f-b887-e50e645c7103 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:51.859087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:51.860109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:51.860758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:51.861850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:51.869200 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:51.904890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:51.906486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db190e-0a81-47eb-97ab-c80aa7590f06 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:51.924400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1275/2548] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:52.800827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:52.803058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:52.945787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadae705-a984-40f1-9d2e-4d2211a91b44 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:52.956503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:52.957180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:52.957492 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:52.957840 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:52.959835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:52.982876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:52.984658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:52.999138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1274/2549] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:55.640253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fbec3-1e76-4678-88a1-a078d7d6b067 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:55.640709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fbec3-1e76-4678-88a1-a078d7d6b067 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:55.640709 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fbec3-1e76-4678-88a1-a078d7d6b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:55.680605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-421fbec3-1e76-4678-88a1-a078d7d6b067 None None] MFA Rules not processed for user `6883da006d3f4aec8805c74cf5ffae86`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:55.681948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-421fbec3-1e76-4678-88a1-a078d7d6b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:55.768757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1276/2550] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:55.782223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:55.795946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454fda-fa0c-4500-9eff-458caedd5109 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:55.802559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:55.803565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:55.804060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:55.804541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:55.814960 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:55.817819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:55.828803 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1275/2551] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:56.027207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:56.027725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:56.028165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:56.049392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:56.050998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:56.121262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1277/2552] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:56.135658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:56.137107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:56.181925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:36:56.216313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093af23d-5ac2-4118-9ee9-6a125290e041 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:56.221810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6edc7-ae34-4f92-bdc8-ba9adf6c5440 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:56.222484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6edc7-ae34-4f92-bdc8-ba9adf6c5440 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:56.222931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6edc7-ae34-4f92-bdc8-ba9adf6c5440 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:56.223565 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6edc7-ae34-4f92-bdc8-ba9adf6c5440 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:56.246718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6edc7-ae34-4f92-bdc8-ba9adf6c5440 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:56.248377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6edc7-ae34-4f92-bdc8-ba9adf6c5440 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:56.258736 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1276/2553] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:36:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:36:56.980527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:56.982003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:57.014225 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e132609-a5db-4ece-895f-2e605852a584 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:57.034320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:57.035584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:57.036103 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:57.036629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:57.039471 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:57.055708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:57.058969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:57.074723 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1278/2554] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:36:57.664977 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d961a-408f-45f9-be58-152939aa9802 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:57.667230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d961a-408f-45f9-be58-152939aa9802 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:57.667610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d961a-408f-45f9-be58-152939aa9802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:57.706759 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-830d961a-408f-45f9-be58-152939aa9802 None None] MFA Rules not processed for user `6883da006d3f4aec8805c74cf5ffae86`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:36:57.708097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-830d961a-408f-45f9-be58-152939aa9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:57.716370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1277/2555] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:36:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:36:58.165400 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:36:58.177926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0403c0-647c-47e5-a241-03b31f128b8c service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:36:58.182046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:36:58.182941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:36:58.183432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:36:58.183930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:36:58.186922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:36:58.198578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:36:58.200032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:36:58.221013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1279/2556] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:36:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:37:06.442551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d961a-408f-45f9-be58-152939aa9802 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:06.444948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-830d961a-408f-45f9-be58-152939aa9802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:06.465632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d961a-408f-45f9-be58-152939aa9802 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:06.470481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:06.471395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:06.471686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:06.471971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:06.474029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:06.548228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:06.549877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636f77a-824c-42e2-9a56-88929ce0cee3 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:06.560935 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1278/2557] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:37:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:37:08.205500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=73 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:37:08.557515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:08.575305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685c7f4-6aef-4a5b-accb-9cbe946329a0 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:08.582267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:08.583170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:08.583450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:08.584057 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:08.586783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:08.604271 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:08.606528 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b5820-7a19-4b0b-a5dc-a176362b48d4 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:08.621649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1280/2558] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:37:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:37:13.502355 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001c3a7-20d6-459f-9165-621a3b339b07 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.502958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001c3a7-20d6-459f-9165-621a3b339b07 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.502958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001c3a7-20d6-459f-9165-621a3b339b07 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.504535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1279/2559] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:37:13.511221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5da7ee-ee2d-4637-b10c-84bdf963d200 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.511704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5da7ee-ee2d-4637-b10c-84bdf963d200 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.511820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5da7ee-ee2d-4637-b10c-84bdf963d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.532999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-1b5da7ee-ee2d-4637-b10c-84bdf963d200 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:37:13.534277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-1b5da7ee-ee2d-4637-b10c-84bdf963d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:13.544972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1281/2560] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:37:13.557173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001c3a7-20d6-459f-9165-621a3b339b07 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:13.558982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a001c3a7-20d6-459f-9165-621a3b339b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:13.623694 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001c3a7-20d6-459f-9165-621a3b339b07 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:13.628546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eeeafc-0465-447e-b3d5-c9473f0cd3bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:13.629247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeeafc-0465-447e-b3d5-c9473f0cd3bc service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.629587 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeeafc-0465-447e-b3d5-c9473f0cd3bc service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.629879 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeeafc-0465-447e-b3d5-c9473f0cd3bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.641087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eeeafc-0465-447e-b3d5-c9473f0cd3bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:13.643728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eeeafc-0465-447e-b3d5-c9473f0cd3bc service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:13.655464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1280/2561] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:37:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:37:13.727187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205751a6-6709-44cd-8059-60ee80d0b7f6 None None] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.727479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205751a6-6709-44cd-8059-60ee80d0b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.727761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205751a6-6709-44cd-8059-60ee80d0b7f6 None None] PATH_INFO: `/` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.729176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1282/2562] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:37:13.733891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551eb14-af54-4abd-b8b1-105f050048ab None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.734180 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551eb14-af54-4abd-b8b1-105f050048ab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.734454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551eb14-af54-4abd-b8b1-105f050048ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.753388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7551eb14-af54-4abd-b8b1-105f050048ab None None] MFA Rules not processed for user `84609c4a913144e2a71c508e96c53d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:37:13.754612 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7551eb14-af54-4abd-b8b1-105f050048ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:13.761336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1281/2563] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:37:13.775727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205751a6-6709-44cd-8059-60ee80d0b7f6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:13.788610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205751a6-6709-44cd-8059-60ee80d0b7f6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:13.793058 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874799cb-6879-43d6-adb6-021d8b10cf7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:13.793740 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799cb-6879-43d6-adb6-021d8b10cf7e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.794031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799cb-6879-43d6-adb6-021d8b10cf7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.794306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799cb-6879-43d6-adb6-021d8b10cf7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.802737 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874799cb-6879-43d6-adb6-021d8b10cf7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:13.804176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874799cb-6879-43d6-adb6-021d8b10cf7e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:13.812109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1283/2564] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:37:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:37:13.908600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa151f9b-2380-4947-9a05-283a83a1c7d1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.908949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa151f9b-2380-4947-9a05-283a83a1c7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.909249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa151f9b-2380-4947-9a05-283a83a1c7d1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.910674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1282/2565] 10.222.0.24 () {54 vars in 921 bytes} [Thu May 19 13:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 13:37:13.916351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644b436-27ee-40ac-b23d-0dfbbd1d0af6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.916642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644b436-27ee-40ac-b23d-0dfbbd1d0af6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.916939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644b436-27ee-40ac-b23d-0dfbbd1d0af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.937257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3644b436-27ee-40ac-b23d-0dfbbd1d0af6 None None] MFA Rules not processed for user `862f9782f8574b8bb99d38c51333277f`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:37:13.938508 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3644b436-27ee-40ac-b23d-0dfbbd1d0af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:13.949102 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1284/2566] 10.222.0.24 () {58 vars in 1018 bytes} [Thu May 19 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:37:13.964032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa151f9b-2380-4947-9a05-283a83a1c7d1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:13.977570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa151f9b-2380-4947-9a05-283a83a1c7d1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:13.981383 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:13.982081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:13.982367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:13.982642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:13.993112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:13.994601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:14.005522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1283/2567] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:37:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:37:15.870198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739af93-4183-441c-bddb-edaa7af7836b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:15.870711 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739af93-4183-441c-bddb-edaa7af7836b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:15.870836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739af93-4183-441c-bddb-edaa7af7836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:15.890001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3739af93-4183-441c-bddb-edaa7af7836b None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:37:15.891408 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3739af93-4183-441c-bddb-edaa7af7836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:15.899363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1285/2568] 10.222.0.24 () {58 vars in 1009 bytes} [Thu May 19 13:37:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:37:15.912421 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:15.924255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908bd14-1cd4-40ff-9eb6-e30685a24001 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:15.929758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:15.930663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:15.930663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:15.930911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:15.940161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:15.941809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:15.950334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1284/2569] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:37:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:37:25.884411 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=21 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:37:25.946933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:37:34.473631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:34.474106 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:34.474208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:34.493155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] MFA Rules not processed for user `e4d792d3845649cf9f28c529b67a2188`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:37:34.494415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:37:34.502119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1286/2570] 10.222.0.24 () {58 vars in 967 bytes} [Thu May 19 13:37:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:37:34.514773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:37:34.525937 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69a7e-fd03-438d-a219-01078b2cd061 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:37:34.531964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78b2f9c966e24c46a23631b80a578862', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:37:34.532615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:37:34.532893 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:37:34.533159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:37:34.542070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:37:34.543684 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:37:34.552361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1285/2571] 10.222.0.24 () {60 vars in 1384 bytes} [Thu May 19 13:37:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 13:37:44.487984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:37:44.549206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:15.570959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:15.581607 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f82a1-9ad9-4d14-bae0-172ac12c8cb1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:15.586122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:15.586766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:15.587052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:15.587299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:15.589247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:15.602049 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:15.603473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:15.614018 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1287/2572] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:38:18.865763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:18.867380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:18.888347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a707161-6d2c-4199-a6c8-23f5b22f25e7 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:18.893679 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:18.894553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:18.894845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:18.895120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] PATH_INFO: `/v3/users/14da69798e344e70ba64bf8187523e45` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:18.896415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14da69798e344e70ba64bf8187523e45)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:18.898515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:18.950198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14da69798e344e70ba64bf8187523e45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:18.961975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14da69798e344e70ba64bf8187523e45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:18.962601 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14da69798e344e70ba64bf8187523e45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:18.963205 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14da69798e344e70ba64bf8187523e45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:18.970465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75dd4e03-8509-4232-b39c-2c5bd2585901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14da69798e344e70ba64bf8187523e45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:18.985175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1286/2573] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:38:18 2022] DELETE /identity/v3/users/14da69798e344e70ba64bf8187523e45 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:38:19.166162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.177303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84440580-6476-47f8-a0b7-7f49ab8581cc service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.241755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.242640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.242925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.243198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] PATH_INFO: `/v3/projects/959c5c84bd724d2688f435b15b41f1c8` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.244396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=959c5c84bd724d2688f435b15b41f1c8)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.246369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.294708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '959c5c84bd724d2688f435b15b41f1c8', 'request_id': 'req-4714bc97-fea7-4c68-8a4a-e2a1d1351639'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:19.304488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '959c5c84bd724d2688f435b15b41f1c8', 'request_id': 'req-4714bc97-fea7-4c68-8a4a-e2a1d1351639'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:19.309915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '959c5c84bd724d2688f435b15b41f1c8', 'request_id': 'req-4714bc97-fea7-4c68-8a4a-e2a1d1351639'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:19.321245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] The token cache is being invalidate because project 959c5c84bd724d2688f435b15b41f1c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:38:19.321605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:19.322484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4714bc97-fea7-4c68-8a4a-e2a1d1351639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:19.324144 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1288/2574] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:38:19 2022] DELETE /identity/v3/projects/959c5c84bd724d2688f435b15b41f1c8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:38:19.329549 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.329847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.330135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.350784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:38:19.351978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:19.415432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1287/2575] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:38:19.424547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.424856 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.425126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.445578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:38:19.446807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:19.455829 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1289/2576] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:38:19.460176 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.470702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1be8fd-b1b9-4597-953d-70d67bfa7142 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.475913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.476572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.476967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.477245 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.478001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:38:19.478304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.479911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.492017 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1288/2577] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:38:19.496881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.508037 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95633fbe-b5bd-47da-9d59-461cc93a6235 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.513349 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.514032 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.514333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.514828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.515943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.519463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.559721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1290/2578] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:38:19 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:38:19.566169 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.580685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7865-5e2a-4c71-9cc4-175b1d71167f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.658567 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.659242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.659518 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.659789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.660480 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.662443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.669179 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:19.687173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1289/2579] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:38:19 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:38:19.691399 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.702959 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ffa20-6c25-4245-8987-13934271d3e6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.707688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.708359 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.708640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.708917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.709826 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.711418 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.725128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1291/2580] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:38:19.726908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.738128 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ff26f-f025-4978-b1d8-560aa499f0d1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.743564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cd9b1-c72c-4afe-bdc5-1c9598bb0e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.744233 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd9b1-c72c-4afe-bdc5-1c9598bb0e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.744510 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd9b1-c72c-4afe-bdc5-1c9598bb0e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.744779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cd9b1-c72c-4afe-bdc5-1c9598bb0e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.745542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cd9b1-c72c-4afe-bdc5-1c9598bb0e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.747158 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cd9b1-c72c-4afe-bdc5-1c9598bb0e6d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.758341 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1290/2581] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:38:19.762732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.774038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b6855-3e2b-465c-b2d0-a314c6239cb6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:19.779366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:19.780382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.780921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.781040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] PATH_INFO: `/v3/projects/10df845b3cb94919a42b8eaf60219f5d/users/b9895b37039747978f5a3d20e9ff63ac/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.800208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10df845b3cb94919a42b8eaf60219f5d, user_id=b9895b37039747978f5a3d20e9ff63ac, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:19.802917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:19.817159 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1292/2582] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:38:19 2022] PUT /identity/v3/projects/10df845b3cb94919a42b8eaf60219f5d/users/b9895b37039747978f5a3d20e9ff63ac/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:38:19.828517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:19.828916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:19.829307 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:19.857712 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] MFA Rules not processed for user `b9895b37039747978f5a3d20e9ff63ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:38:19.859093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:19.924764 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1291/2583] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:38:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:38:19.929952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:19.938641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:20.006185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caca9bb-59d1-4fcc-bb0a-f41b7d5f968d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.012836 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.013877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.014215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.014742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.015529 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.017511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.046609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1293/2584] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:38:19 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:38:20.050799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.063023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c482425-0e8c-4905-84d6-c23dacac131c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.139223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.140520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.140807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.141086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.141871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.143880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.165931 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1292/2585] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:38:20 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:38:20.170494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.181161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ca1c4-84be-4490-8090-32f6f0cb1c98 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.185802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.186527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.187017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.187222 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.187964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.189609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.200273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1294/2586] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:38:20.204749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.215862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ceab09-f831-45bb-aca3-218bd2683673 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.222026 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.222952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.223232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.223508 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] PATH_INFO: `/v3/projects/4771139ec59248f382952535b8ba4110/users/f458e1549daf48ffbc195841f0033fdc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.240454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4771139ec59248f382952535b8ba4110, user_id=f458e1549daf48ffbc195841f0033fdc, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.243219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.259157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1293/2587] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:38:20 2022] PUT /identity/v3/projects/4771139ec59248f382952535b8ba4110/users/f458e1549daf48ffbc195841f0033fdc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:38:20.263553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.274760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed451f9c-a199-42d0-863f-dd9d38ff4bb2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.335630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988c1b8-adee-4e69-b83c-f6140145fdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.336303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988c1b8-adee-4e69-b83c-f6140145fdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.336622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988c1b8-adee-4e69-b83c-f6140145fdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.336908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988c1b8-adee-4e69-b83c-f6140145fdd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.337690 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988c1b8-adee-4e69-b83c-f6140145fdd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.339375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988c1b8-adee-4e69-b83c-f6140145fdd9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.350060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1295/2588] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:38:20.354578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.366231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fba61e-c6e9-49d7-b87d-e3af6e183a5c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.371504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.372389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.372676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.372966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] PATH_INFO: `/v3/projects/4771139ec59248f382952535b8ba4110/users/f458e1549daf48ffbc195841f0033fdc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.375762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4771139ec59248f382952535b8ba4110, user_id=f458e1549daf48ffbc195841f0033fdc, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.378478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.390288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1294/2589] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:38:20 2022] PUT /identity/v3/projects/4771139ec59248f382952535b8ba4110/users/f458e1549daf48ffbc195841f0033fdc/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:38:20.400443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.400730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.400999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.429190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] MFA Rules not processed for user `f458e1549daf48ffbc195841f0033fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:38:20.430438 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:20.496742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:20.500832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1296/2590] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:38:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:38:20.512699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.514770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:20.538464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:20.575767 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcdf21-acc2-4278-8723-9adfa17b977f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.580659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.581308 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.581627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.581912 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.591676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.593779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.603522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1295/2591] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:38:20.686321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:20.696636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aed3ca-58b3-4c6a-b674-c9f680e44a0c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:20.700719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392c0ae-245b-4be6-859a-1c40b25d2cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:20.701360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392c0ae-245b-4be6-859a-1c40b25d2cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:20.701680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392c0ae-245b-4be6-859a-1c40b25d2cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:20.702122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392c0ae-245b-4be6-859a-1c40b25d2cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:20.761316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392c0ae-245b-4be6-859a-1c40b25d2cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:20.762714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392c0ae-245b-4be6-859a-1c40b25d2cd2 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:20.771712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1297/2592] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:38:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:38:30.599498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=17 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:30.767864 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=28 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:38:59.418583 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:59.481476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2f1ad-beda-4628-bbf8-fbd4c8a45520 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:59.486428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:59.487305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:59.487581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:59.487849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] PATH_INFO: `/v3/users/b9895b37039747978f5a3d20e9ff63ac` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:59.489056 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9895b37039747978f5a3d20e9ff63ac)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:59.491173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:59.546783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9895b37039747978f5a3d20e9ff63ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:59.559359 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9895b37039747978f5a3d20e9ff63ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:59.559985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9895b37039747978f5a3d20e9ff63ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:59.560582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9895b37039747978f5a3d20e9ff63ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:59.567905 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9895b37039747978f5a3d20e9ff63ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:38:59.582236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1296/2593] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:38:59 2022] DELETE /identity/v3/users/b9895b37039747978f5a3d20e9ff63ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:38:59.587095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a252-f9c8-432c-935f-17650550d176 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:59.587639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a252-f9c8-432c-935f-17650550d176 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:59.587639 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a252-f9c8-432c-935f-17650550d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:59.607337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-af89a252-f9c8-432c-935f-17650550d176 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:38:59.608498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af89a252-f9c8-432c-935f-17650550d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:59.680110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1298/2594] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:38:59.691969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:38:59.693389 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:38:59.751338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a774fad-fbd4-4ac4-9e77-f0748ae66781 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:38:59.755027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:38:59.755668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:38:59.755948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:38:59.756226 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:38:59.765668 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:38:59.767135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:38:59.777993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1297/2595] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:00.341910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a252-f9c8-432c-935f-17650550d176 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:00.343525 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-af89a252-f9c8-432c-935f-17650550d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:00.356142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a252-f9c8-432c-935f-17650550d176 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:00.361440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:00.362344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:00.362772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:00.362916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] PATH_INFO: `/v3/projects/10df845b3cb94919a42b8eaf60219f5d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:00.364124 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10df845b3cb94919a42b8eaf60219f5d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:00.366886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:00.416083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10df845b3cb94919a42b8eaf60219f5d', 'request_id': 'req-118bc6c7-fcaa-462e-81df-e6f0c4552e41'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.426163 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10df845b3cb94919a42b8eaf60219f5d', 'request_id': 'req-118bc6c7-fcaa-462e-81df-e6f0c4552e41'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.431922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10df845b3cb94919a42b8eaf60219f5d', 'request_id': 'req-118bc6c7-fcaa-462e-81df-e6f0c4552e41'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.443257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] The token cache is being invalidate because project 10df845b3cb94919a42b8eaf60219f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:00.443550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.444263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.445904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1299/2596] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:00 2022] DELETE /identity/v3/projects/10df845b3cb94919a42b8eaf60219f5d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:00.450371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:00.460664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb10e2-cef1-48d7-b6ac-b9155d60b0c0 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:00.484625 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:00.526259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:00.527116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:00.527423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:00.527708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] PATH_INFO: `/v3/users/f458e1549daf48ffbc195841f0033fdc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:00.528948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f458e1549daf48ffbc195841f0033fdc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:00.530911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:00.582451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f458e1549daf48ffbc195841f0033fdc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.594841 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f458e1549daf48ffbc195841f0033fdc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.595514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f458e1549daf48ffbc195841f0033fdc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.596132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f458e1549daf48ffbc195841f0033fdc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.603106 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4e5a374e-673e-4fe0-b35c-e64003ce3d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f458e1549daf48ffbc195841f0033fdc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:00.617610 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1298/2597] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:00 2022] DELETE /identity/v3/users/f458e1549daf48ffbc195841f0033fdc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:00.935165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:00.936669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:00.950779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bc6c7-fcaa-462e-81df-e6f0c4552e41 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.013035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.013935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.014216 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.014481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] PATH_INFO: `/v3/projects/4771139ec59248f382952535b8ba4110` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.015745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4771139ec59248f382952535b8ba4110)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.017669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.054927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:01.073424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4771139ec59248f382952535b8ba4110', 'request_id': 'req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:01.081330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4771139ec59248f382952535b8ba4110', 'request_id': 'req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:01.087059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4771139ec59248f382952535b8ba4110', 'request_id': 'req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:01.098533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] The token cache is being invalidate because project 4771139ec59248f382952535b8ba4110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:01.098819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:01.099577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-7d960ac0-d1bf-468c-88e8-8d42cad9f93c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:01.101262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1300/2598] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:00 2022] DELETE /identity/v3/projects/4771139ec59248f382952535b8ba4110 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:01.106544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.106827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.107103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.125762 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:01.126964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:01.203813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1299/2599] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:01.211800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.212092 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.212372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.231946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:01.233292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:01.242802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1301/2600] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:01.247924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.260362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4ee35-faf4-4ecf-b606-073caea4c605 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.266428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.267095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.267378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.267647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.268379 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:01.268685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.270342 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.283272 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1300/2601] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:01.288035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.297997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcc747-f4ca-4ddb-b09d-554fa7eb35c0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.303187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.303877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.304161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.304436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.305161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.307148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.334396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1302/2602] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:01.338666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.348957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d7f8-d226-46af-b4a2-671495cb0640 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.376544 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:01.409024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.409776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.410061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.410335 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.411113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.413119 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.437263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1301/2603] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:01.441745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.453120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0a8db-be76-47e8-971d-2efa719914c6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.458214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.458816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.459100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.459380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.460136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.461831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.473040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1303/2604] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:01.477975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.490264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9fdf2-2e75-478e-bf18-6925c650671e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.496178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864f726-ab3c-462f-8438-1158bb977fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.497102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f726-ab3c-462f-8438-1158bb977fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.497388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f726-ab3c-462f-8438-1158bb977fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.497699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f726-ab3c-462f-8438-1158bb977fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.498441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864f726-ab3c-462f-8438-1158bb977fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.500065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864f726-ab3c-462f-8438-1158bb977fcf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.510486 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1302/2605] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:01.515277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.525893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236bf4c-b27d-4d1d-ac53-e5297da32d25 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.531526 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.532424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.532702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.533000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] PATH_INFO: `/v3/projects/21b7b304705c4d4fbad4dfa3dea6d374/users/9102b347678d462fba762844b0cec4dc/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.551766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b7b304705c4d4fbad4dfa3dea6d374, user_id=9102b347678d462fba762844b0cec4dc, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.554477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.572353 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1304/2606] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:01 2022] PUT /identity/v3/projects/21b7b304705c4d4fbad4dfa3dea6d374/users/9102b347678d462fba762844b0cec4dc/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:01.583721 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.584230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.584666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.612732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] MFA Rules not processed for user `9102b347678d462fba762844b0cec4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:01.613992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:01.686490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1303/2607] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:39:01.691304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.755563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85d04c-573c-451d-99f6-56a8a7f16065 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.760609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.761261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.761676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.761983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.762674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.764559 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.797620 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1305/2608] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:01.803804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.815367 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfba739-5fad-4cdc-87c6-095c89049567 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.871277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.871953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.872232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.872505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.873183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.875146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.896516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1304/2609] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:01 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:39:01.900916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.912909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baaf7f-77d4-4d23-b3e5-7d5928ccf49d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.918653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.919784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.920072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.920354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.921425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.923031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.936560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1306/2610] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:01.940132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:01.951413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c0287-573a-439e-8a3d-ef696cd9138a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:01.956416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:01.957388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:01.957773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:01.958120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] PATH_INFO: `/v3/projects/dc5a93a26c04403793af839f1f60cf99/users/77c0642a21244219b2b58034742d91a6/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:01.974454 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5a93a26c04403793af839f1f60cf99, user_id=77c0642a21244219b2b58034742d91a6, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:01.977252 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:01.992345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1305/2611] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:01 2022] PUT /identity/v3/projects/dc5a93a26c04403793af839f1f60cf99/users/77c0642a21244219b2b58034742d91a6/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:01.996446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:02.007581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5d360-3de5-4323-a544-5748c5bfef60 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:02.028595 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:02.073971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e58e48-3ec8-4fad-9327-5642c602b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:02.074831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e58e48-3ec8-4fad-9327-5642c602b503 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.075112 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e58e48-3ec8-4fad-9327-5642c602b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.075388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e58e48-3ec8-4fad-9327-5642c602b503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.076123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e58e48-3ec8-4fad-9327-5642c602b503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:02.077753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e58e48-3ec8-4fad-9327-5642c602b503 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:02.087389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1307/2612] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:02.091697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:02.097232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:02.102746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f382c7-8ecd-44c1-8cb1-694c9da8d420 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:02.108447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:02.109332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.109646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.109924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] PATH_INFO: `/v3/projects/dc5a93a26c04403793af839f1f60cf99/users/77c0642a21244219b2b58034742d91a6/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.111958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5a93a26c04403793af839f1f60cf99, user_id=77c0642a21244219b2b58034742d91a6, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:02.114645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:02.125005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1306/2613] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:02 2022] PUT /identity/v3/projects/dc5a93a26c04403793af839f1f60cf99/users/77c0642a21244219b2b58034742d91a6/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:39:02.136259 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.136547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.136857 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.167423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] MFA Rules not processed for user `77c0642a21244219b2b58034742d91a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:02.169196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:02.242491 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1308/2614] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:02.254749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:02.256870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:02.324140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30624ce-52c1-4fe2-9bfe-e4cb7ab24539 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:02.328783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce3bdc-ec89-4b89-bca8-804e084f74f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:02.329526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3bdc-ec89-4b89-bca8-804e084f74f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.329787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3bdc-ec89-4b89-bca8-804e084f74f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.330053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce3bdc-ec89-4b89-bca8-804e084f74f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.341034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce3bdc-ec89-4b89-bca8-804e084f74f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:02.342538 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce3bdc-ec89-4b89-bca8-804e084f74f7 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:02.353058 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1307/2615] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:02.381667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:02.447338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d650b3-598a-4b9b-a1d9-bb491e8093ce None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:02.453443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:02.454501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.454782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.455061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] PATH_INFO: `/v3/users/9102b347678d462fba762844b0cec4dc` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.456490 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9102b347678d462fba762844b0cec4dc)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:02.458502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:02.508420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9102b347678d462fba762844b0cec4dc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:02.510120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:02.519250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9102b347678d462fba762844b0cec4dc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:02.520272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9102b347678d462fba762844b0cec4dc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:02.521009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9102b347678d462fba762844b0cec4dc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:02.533023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9102b347678d462fba762844b0cec4dc'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:02.547013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1309/2616] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:02 2022] DELETE /identity/v3/users/9102b347678d462fba762844b0cec4dc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:02.551894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80458a16-8456-4476-a990-d372257db2e9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.552414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80458a16-8456-4476-a990-d372257db2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.552414 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80458a16-8456-4476-a990-d372257db2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.571107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-80458a16-8456-4476-a990-d372257db2e9 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:02.572332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-80458a16-8456-4476-a990-d372257db2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:02.636457 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1308/2617] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:02.646484 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:02.647950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:02.705882 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238fefd-035f-48e9-8196-bab2e124a885 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:02.709863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:02.710521 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:02.710800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:02.711073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:02.721775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:02.723229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:02.733646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1310/2618] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:39:03.015708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80458a16-8456-4476-a990-d372257db2e9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:03.017263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-80458a16-8456-4476-a990-d372257db2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:03.030087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80458a16-8456-4476-a990-d372257db2e9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:03.035452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:03.036331 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.036616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.036891 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] PATH_INFO: `/v3/projects/21b7b304705c4d4fbad4dfa3dea6d374` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.038130 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b7b304705c4d4fbad4dfa3dea6d374)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:03.040073 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:03.089581 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b7b304705c4d4fbad4dfa3dea6d374', 'request_id': 'req-60a2bf87-407f-4604-8b1d-61dbcd3b784d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.098370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b7b304705c4d4fbad4dfa3dea6d374', 'request_id': 'req-60a2bf87-407f-4604-8b1d-61dbcd3b784d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.104947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b7b304705c4d4fbad4dfa3dea6d374', 'request_id': 'req-60a2bf87-407f-4604-8b1d-61dbcd3b784d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.116422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] The token cache is being invalidate because project 21b7b304705c4d4fbad4dfa3dea6d374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:03.116734 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.117425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.119146 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1309/2619] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:03 2022] DELETE /identity/v3/projects/21b7b304705c4d4fbad4dfa3dea6d374 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:03.123582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:03.133894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a386-5244-4337-86be-49d8b0c74181 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:03.203152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:03.204029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.204305 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.204578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] PATH_INFO: `/v3/users/77c0642a21244219b2b58034742d91a6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.205808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c0642a21244219b2b58034742d91a6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:03.207721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:03.262676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c0642a21244219b2b58034742d91a6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.271429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c0642a21244219b2b58034742d91a6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.272080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c0642a21244219b2b58034742d91a6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.272693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c0642a21244219b2b58034742d91a6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.284158 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:03.285561 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-a8f01af1-42ee-4e14-84f9-055d6b130912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c0642a21244219b2b58034742d91a6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.301581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1311/2620] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:03 2022] DELETE /identity/v3/users/77c0642a21244219b2b58034742d91a6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:03.588512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:03.590234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:03.603634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2bf87-407f-4604-8b1d-61dbcd3b784d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:03.617660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:03.668739 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:03.669561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.669845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.670103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] PATH_INFO: `/v3/projects/dc5a93a26c04403793af839f1f60cf99` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.671748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5a93a26c04403793af839f1f60cf99)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:03.673727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:03.724091 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5a93a26c04403793af839f1f60cf99', 'request_id': 'req-c1524bd4-3151-455e-821f-1905927b3e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.733251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5a93a26c04403793af839f1f60cf99', 'request_id': 'req-c1524bd4-3151-455e-821f-1905927b3e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.739667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5a93a26c04403793af839f1f60cf99', 'request_id': 'req-c1524bd4-3151-455e-821f-1905927b3e85'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.750961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] The token cache is being invalidate because project dc5a93a26c04403793af839f1f60cf99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:03.751251 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.751998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c1524bd4-3151-455e-821f-1905927b3e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:03.753733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1310/2621] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:03 2022] DELETE /identity/v3/projects/dc5a93a26c04403793af839f1f60cf99 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:03.759289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.759577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.759852 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.779059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:03.780287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:03.844147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1312/2622] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:03.852411 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.852700 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.852997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.872979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:03.874246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:03.884150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1311/2623] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:03.888576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:03.898799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf92c95-7e8c-42fb-8fd9-012ad58b5612 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:03.905339 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:03.906035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.906337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.906610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.907345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:03.907651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:03.909263 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:03.922514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1313/2624] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:03.928947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:03.940253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085e62c-ff55-4cea-82e9-79a8a2511a76 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:03.946135 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:03.946807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:03.947088 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:03.947369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:03.948092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:03.950261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:03.979184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1312/2625] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:03 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:03.983600 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:03.993929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1a104-7a9c-4ac8-ac7a-e29942fbd857 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.062745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.072184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.072627 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.072827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.073450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.075396 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.102041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1314/2626] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:03 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:04.106569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.119313 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a54a6-b4ab-426b-b5c8-b579df17a459 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.124605 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.125916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.126206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.126466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.127269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.129590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.141293 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1313/2627] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:04.145582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.156638 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2399c1b-5c1e-4d20-a254-754636560912 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.165013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d6783-61f8-4073-b7d2-5e9bd0daf5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.166039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d6783-61f8-4073-b7d2-5e9bd0daf5df admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.166430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d6783-61f8-4073-b7d2-5e9bd0daf5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.166774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d6783-61f8-4073-b7d2-5e9bd0daf5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.167892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d6783-61f8-4073-b7d2-5e9bd0daf5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.170377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d6783-61f8-4073-b7d2-5e9bd0daf5df admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.185839 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1315/2628] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:04.189997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.200578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c41af8-9cfd-47c8-b3e9-9bff6e0f523a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.206468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.207413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.207695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.207969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] PATH_INFO: `/v3/projects/621cae2a65da407d928cc536c8cc19ed/users/6e7ca8982b3745eab64f674b2bd41ce6/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.223553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] RBAC: Authorizing `identity:create_grant(project_id=621cae2a65da407d928cc536c8cc19ed, user_id=6e7ca8982b3745eab64f674b2bd41ce6, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.226282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.242674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1314/2629] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:04 2022] PUT /identity/v3/projects/621cae2a65da407d928cc536c8cc19ed/users/6e7ca8982b3745eab64f674b2bd41ce6/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:04.253076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.253371 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.253696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.281547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] MFA Rules not processed for user `6e7ca8982b3745eab64f674b2bd41ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:04.282766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:04.326504 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:04.350846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1316/2630] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:04.355491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.403246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:04.425332 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269afaa8-e56f-4c74-bc38-bde2c761c93a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.430029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61830-3030-4024-9539-a96b608571df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.430678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61830-3030-4024-9539-a96b608571df admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.430949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61830-3030-4024-9539-a96b608571df admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.431231 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61830-3030-4024-9539-a96b608571df admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.431917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b61830-3030-4024-9539-a96b608571df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.434174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b61830-3030-4024-9539-a96b608571df admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.460586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1315/2631] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:04 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:04.464672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.475785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a7519-6e72-4128-84c2-b5d07e33f74c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.548675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.549344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.549660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.549938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.550629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.552553 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.575808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1317/2632] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:04 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:04.579860 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61830-3030-4024-9539-a96b608571df admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.590665 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61830-3030-4024-9539-a96b608571df admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.595590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.596269 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.596547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.596845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.597606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.599204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.610211 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1316/2633] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:04.614427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.624803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a99919-6134-48bf-9a87-9ede32d4179e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.630485 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.631379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.631666 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.631945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] PATH_INFO: `/v3/projects/4cb49a5864184938a340ddd14825ec95/users/8b3fb805d8584fd483ebdfa9747a902a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.648653 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb49a5864184938a340ddd14825ec95, user_id=8b3fb805d8584fd483ebdfa9747a902a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.651389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.666134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1318/2634] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:04 2022] PUT /identity/v3/projects/4cb49a5864184938a340ddd14825ec95/users/8b3fb805d8584fd483ebdfa9747a902a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:04.670281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.681203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834566e3-d92d-4a4b-8a00-f566e3ef5453 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.742126 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dc464-8f33-4776-87bd-a15526ed9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.742862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dc464-8f33-4776-87bd-a15526ed9b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.743148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dc464-8f33-4776-87bd-a15526ed9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.743447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dc464-8f33-4776-87bd-a15526ed9b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.744217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dc464-8f33-4776-87bd-a15526ed9b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.745933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dc464-8f33-4776-87bd-a15526ed9b50 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.757062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1317/2635] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:04.761208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.772425 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216997-10c0-43a7-b96f-b36751f41015 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.777706 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.778602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.778919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.779197 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] PATH_INFO: `/v3/projects/4cb49a5864184938a340ddd14825ec95/users/8b3fb805d8584fd483ebdfa9747a902a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.781237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb49a5864184938a340ddd14825ec95, user_id=8b3fb805d8584fd483ebdfa9747a902a, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:04.783982 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:04.794890 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1319/2636] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:04 2022] PUT /identity/v3/projects/4cb49a5864184938a340ddd14825ec95/users/8b3fb805d8584fd483ebdfa9747a902a/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:04.807309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.807602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.807971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:04.838555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] MFA Rules not processed for user `8b3fb805d8584fd483ebdfa9747a902a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:04.839797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:04.914013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1318/2637] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:39:04.926113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:04.928230 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:04.993250 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ec1ec-8a13-4080-99e0-bb4f59d1b877 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:04.997381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:04.998097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:04.998380 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:04.998656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:05.008985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:05.010524 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:05.020800 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1320/2638] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:05.069327 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:05.071067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:05.084582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe1a31-41a1-43b6-8c46-5de9f83d22b8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:05.088997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7f925-e409-40a5-a388-7f22584d6fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:05.089672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7f925-e409-40a5-a388-7f22584d6fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:05.089980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7f925-e409-40a5-a388-7f22584d6fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:05.090277 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7f925-e409-40a5-a388-7f22584d6fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:05.125304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:05.150926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7f925-e409-40a5-a388-7f22584d6fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:05.152428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7f925-e409-40a5-a388-7f22584d6fe5 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:05.161979 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1319/2639] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:08.501833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:08.544958 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:08.568724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66da58-c750-4453-b728-b27497de16b1 service cinder] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:08.574061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:08.574936 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:08.575220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:08.575495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] PATH_INFO: `/v3/users/6e7ca8982b3745eab64f674b2bd41ce6` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:08.576938 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e7ca8982b3745eab64f674b2bd41ce6)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:08.578953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:08.628996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e7ca8982b3745eab64f674b2bd41ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:08.636934 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e7ca8982b3745eab64f674b2bd41ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:08.637696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e7ca8982b3745eab64f674b2bd41ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:08.638318 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e7ca8982b3745eab64f674b2bd41ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:08.651993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e7ca8982b3745eab64f674b2bd41ce6'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:08.665629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1321/2640] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:08 2022] DELETE /identity/v3/users/6e7ca8982b3745eab64f674b2bd41ce6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:08.670186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:08.671101 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:08.671217 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:08.688773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:08.690002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:08.749632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1320/2641] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:08.760073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:08.761574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:08.819352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b12df-9d31-4fe3-aa02-a2f58e5bcfaa admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:08.823739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:08.824697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:08.825100 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:08.825356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:08.835013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:08.836444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:08.846185 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1322/2642] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 13:39:09.119209 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:09.120991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:09.134166 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737d264-8d1b-44d9-ae27-595e9400fe28 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:09.139320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:09.140320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:09.140614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:09.140942 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] PATH_INFO: `/v3/projects/621cae2a65da407d928cc536c8cc19ed` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:09.142752 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] RBAC: Authorizing `identity:delete_project(project_id=621cae2a65da407d928cc536c8cc19ed)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:09.144141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:09.206500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621cae2a65da407d928cc536c8cc19ed', 'request_id': 'req-7ace111b-f0af-4225-8727-e58638e4432e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.218425 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621cae2a65da407d928cc536c8cc19ed', 'request_id': 'req-7ace111b-f0af-4225-8727-e58638e4432e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.225637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621cae2a65da407d928cc536c8cc19ed', 'request_id': 'req-7ace111b-f0af-4225-8727-e58638e4432e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.238516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] The token cache is being invalidate because project 621cae2a65da407d928cc536c8cc19ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:09.238809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.243434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.245741 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1321/2643] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:09 2022] DELETE /identity/v3/projects/621cae2a65da407d928cc536c8cc19ed => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:09.250253 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:09.260126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd2037-d66d-45d9-8b7c-6f2f43df7354 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:09.319029 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:09.319905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:09.320180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:09.320448 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] PATH_INFO: `/v3/users/8b3fb805d8584fd483ebdfa9747a902a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:09.321784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3fb805d8584fd483ebdfa9747a902a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:09.323808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:09.325435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:09.381503 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3fb805d8584fd483ebdfa9747a902a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.391080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3fb805d8584fd483ebdfa9747a902a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.391697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3fb805d8584fd483ebdfa9747a902a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.392309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3fb805d8584fd483ebdfa9747a902a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.410495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-d205fde9-3301-4e8c-81cd-fc03119abdc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3fb805d8584fd483ebdfa9747a902a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:09.424715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1323/2644] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:09 2022] DELETE /identity/v3/users/8b3fb805d8584fd483ebdfa9747a902a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:10.045324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.047783 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:10.061947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace111b-f0af-4225-8727-e58638e4432e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.132787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.134235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.134599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.134947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] PATH_INFO: `/v3/projects/4cb49a5864184938a340ddd14825ec95` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.136265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb49a5864184938a340ddd14825ec95)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.138320 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.140122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:10.200213 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb49a5864184938a340ddd14825ec95', 'request_id': 'req-363faa3b-81d9-4753-b210-a4e7b9886ab2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:10.211273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb49a5864184938a340ddd14825ec95', 'request_id': 'req-363faa3b-81d9-4753-b210-a4e7b9886ab2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:10.223047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb49a5864184938a340ddd14825ec95', 'request_id': 'req-363faa3b-81d9-4753-b210-a4e7b9886ab2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:10.236400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] The token cache is being invalidate because project 4cb49a5864184938a340ddd14825ec95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:10.236750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:10.237593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-363faa3b-81d9-4753-b210-a4e7b9886ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:10.244230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1322/2645] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:10 2022] DELETE /identity/v3/projects/4cb49a5864184938a340ddd14825ec95 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:10.249907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.250381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.250544 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.270708 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:10.272228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:10.340494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1324/2646] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:10.348475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.348852 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.349200 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.368983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:10.370134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:10.378458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1323/2647] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:10.382838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.392682 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b129d4-8b6c-40eb-a4da-8df56dad86a3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.398535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.399236 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.399517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.399794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.400530 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:10.400835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.402550 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.415268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1325/2648] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:10.423064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.433616 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4c8d-c349-4d1e-b136-4a95e7c2788b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.438732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c76ac-285d-427b-82a4-a814ce50ca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.439370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c76ac-285d-427b-82a4-a814ce50ca36 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.439799 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c76ac-285d-427b-82a4-a814ce50ca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.439948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c76ac-285d-427b-82a4-a814ce50ca36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.440645 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c76ac-285d-427b-82a4-a814ce50ca36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.442641 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c76ac-285d-427b-82a4-a814ce50ca36 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.472152 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1324/2649] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:10.476174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.487589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eab81f-6c13-4562-853b-f865cf65416d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.515381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.515983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.516318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.534940 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] MFA Rules not processed for user `aa0a4b41707d444581ee0aa2296ac0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:10.536164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:10.555739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.556537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.556885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.557170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.557915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.560068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.570904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:10.586714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1326/2650] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:10.590835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.602164 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcd097-8466-4828-aa33-ef8d0e6c651d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.603755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1325/2651] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:39:10.607537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.608505 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.608784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.609060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.609846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.611459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.623016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1327/2652] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:39:10.627742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.638918 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50de07-590d-4197-bad5-78a49d50ebc9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.646790 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8d783-819e-4c3d-ad15-b145f1c88143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.647561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8d783-819e-4c3d-ad15-b145f1c88143 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.647850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8d783-819e-4c3d-ad15-b145f1c88143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.648131 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8d783-819e-4c3d-ad15-b145f1c88143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.648872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8d783-819e-4c3d-ad15-b145f1c88143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.650582 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8d783-819e-4c3d-ad15-b145f1c88143 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.662154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1326/2653] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:10.666372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.677157 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e6475-7e11-4a24-bd16-fb6ed04362b3 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.681838 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.682734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.683013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.683286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] PATH_INFO: `/v3/projects/b1d90fc64b7a41fbb59cf5830a988862/users/24913a6e44534bcdbbdf6b17d238431d/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.700188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d90fc64b7a41fbb59cf5830a988862, user_id=24913a6e44534bcdbbdf6b17d238431d, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.703827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.720515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1328/2654] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:10 2022] PUT /identity/v3/projects/b1d90fc64b7a41fbb59cf5830a988862/users/24913a6e44534bcdbbdf6b17d238431d/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:10.731966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.732262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.732561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.760377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] MFA Rules not processed for user `24913a6e44534bcdbbdf6b17d238431d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:10.761720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:10.808640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:10.835706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1327/2655] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:10.841834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.907406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a966e4-482f-4caa-8eaa-032673a19d05 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:10.912056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:10.912725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:10.913012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:10.913289 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:10.914130 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:10.915983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:10.947363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1329/2656] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:10.948547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:10.960125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad334117-2ae5-4835-ad1c-43baa8010f86 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.021496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.022170 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.022450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.022725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.023433 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.025385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.050424 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1328/2657] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:10 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:39:11.054333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.065202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d361f1-88a3-41e3-8fdb-0c67c0dc31bc admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.070026 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.070703 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.070962 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.071255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.072013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.073658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.084964 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1330/2658] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:11.089067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.099623 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb9dc6-6ab5-43fc-a8fe-bdb99c8b23c6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.104469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6b5e0-4160-4462-94f6-569b03b69225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.105144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b5e0-4160-4462-94f6-569b03b69225 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.105443 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b5e0-4160-4462-94f6-569b03b69225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.105722 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6b5e0-4160-4462-94f6-569b03b69225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.106489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b5e0-4160-4462-94f6-569b03b69225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.108304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6b5e0-4160-4462-94f6-569b03b69225 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.119002 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1329/2659] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:11.123344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.133842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d9108-fd6d-4adf-8677-a204f68f4e27 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.139578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.140567 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.140930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.141202 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] PATH_INFO: `/v3/projects/490d75e43bed49cebc61551fdd29559e/users/9ab6feb90b2a4827bc8a83e425bc425e/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.157215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] RBAC: Authorizing `identity:create_grant(project_id=490d75e43bed49cebc61551fdd29559e, user_id=9ab6feb90b2a4827bc8a83e425bc425e, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.160036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.174660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1331/2660] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:11 2022] PUT /identity/v3/projects/490d75e43bed49cebc61551fdd29559e/users/9ab6feb90b2a4827bc8a83e425bc425e/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:11.185348 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.185676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.185966 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.213495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] MFA Rules not processed for user `9ab6feb90b2a4827bc8a83e425bc425e`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:11.214585 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:11.287063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1330/2661] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:11.291424 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.368582 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860d5d5-7a62-41a1-8e6d-40c4012abede admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.375219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.375911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.376196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.376474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.377209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.379213 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.396660 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:11.413560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1332/2662] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:11 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:11.416174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.427924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21490d9e-8e9f-427f-9eaa-8b8c268b55a2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.500794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:11.512533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.513363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.513664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.513948 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.515373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.517350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.537798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1331/2663] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:11 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:11.541859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.553115 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404febd6-e4b0-4110-9258-6178f45859da admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.559088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.559770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.560034 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.560309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.561152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.563363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.578111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1333/2664] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:11.579498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.589699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a02719-e100-41b2-8970-cccc5b0969e5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.594405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.595296 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.595578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.595851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] PATH_INFO: `/v3/projects/6135948dec2d4fa78a4e0b854ed00f47/users/c0cc436f99ab49a9a70bd99424516c83/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.613054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6135948dec2d4fa78a4e0b854ed00f47, user_id=c0cc436f99ab49a9a70bd99424516c83, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.616413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.629442 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1332/2665] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:11 2022] PUT /identity/v3/projects/6135948dec2d4fa78a4e0b854ed00f47/users/c0cc436f99ab49a9a70bd99424516c83/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:11.633547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.644196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2ce0f-8da8-4297-a36f-3a1955745198 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.712079 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c116a-6d0d-4b9a-8726-0b45882c4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.713019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c116a-6d0d-4b9a-8726-0b45882c4602 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.713296 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c116a-6d0d-4b9a-8726-0b45882c4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.713602 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c116a-6d0d-4b9a-8726-0b45882c4602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.714340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c116a-6d0d-4b9a-8726-0b45882c4602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.715915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c116a-6d0d-4b9a-8726-0b45882c4602 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.730217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1334/2666] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:39:11.731629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.742629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68374de9-41cd-4b36-9988-2643c64d9d41 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.747380 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.748253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.748593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.748921 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] PATH_INFO: `/v3/projects/6135948dec2d4fa78a4e0b854ed00f47/users/c0cc436f99ab49a9a70bd99424516c83/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.760142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6135948dec2d4fa78a4e0b854ed00f47, user_id=c0cc436f99ab49a9a70bd99424516c83, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.760142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.766232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1333/2667] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:11 2022] PUT /identity/v3/projects/6135948dec2d4fa78a4e0b854ed00f47/users/c0cc436f99ab49a9a70bd99424516c83/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:11.776876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df884167-4496-4f38-9030-a90182acf112 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.777138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df884167-4496-4f38-9030-a90182acf112 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.777394 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df884167-4496-4f38-9030-a90182acf112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.806842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-df884167-4496-4f38-9030-a90182acf112 None None] MFA Rules not processed for user `c0cc436f99ab49a9a70bd99424516c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:11.808063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-df884167-4496-4f38-9030-a90182acf112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:11.881177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1335/2668] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:11.892698 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:11.894763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:11.957813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421fc17-a8c3-4eeb-9638-ef17f3826704 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:11.961505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:11.962188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:11.962466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:11.962742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:11.973573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:11.975030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:11.984851 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1334/2669] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:12.025940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df884167-4496-4f38-9030-a90182acf112 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:12.027456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-df884167-4496-4f38-9030-a90182acf112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:12.040406 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df884167-4496-4f38-9030-a90182acf112 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:12.044973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50058ea-0efd-43e0-86f5-f8bec83c2c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:12.046024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50058ea-0efd-43e0-86f5-f8bec83c2c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:12.046302 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50058ea-0efd-43e0-86f5-f8bec83c2c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:12.046574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50058ea-0efd-43e0-86f5-f8bec83c2c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:12.108853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50058ea-0efd-43e0-86f5-f8bec83c2c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:12.110330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50058ea-0efd-43e0-86f5-f8bec83c2c33 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:12.120147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1336/2670] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:15.162458 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:15.231378 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd87901-0a41-4358-95d0-0913391d3733 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:15.236493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:15.237546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:15.237782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:15.238027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] PATH_INFO: `/v3/users/24913a6e44534bcdbbdf6b17d238431d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:15.239244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24913a6e44534bcdbbdf6b17d238431d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:15.241476 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:15.294204 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24913a6e44534bcdbbdf6b17d238431d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.296813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:15.309904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24913a6e44534bcdbbdf6b17d238431d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.310618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24913a6e44534bcdbbdf6b17d238431d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.311309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24913a6e44534bcdbbdf6b17d238431d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.318731 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24913a6e44534bcdbbdf6b17d238431d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.333678 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1335/2671] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:15 2022] DELETE /identity/v3/users/24913a6e44534bcdbbdf6b17d238431d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:15.338732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:15.339201 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:15.339330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:15.346821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:15.357960 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:15.359278 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:15.423622 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1337/2672] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:15.433857 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:15.436038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:15.498066 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b7259-341a-49ed-9a94-b53f408fc77d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:15.502685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:15.503361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:15.503642 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:15.503916 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:15.514164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:15.515742 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:15.525521 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1336/2673] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:15.815847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:15.817818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:15.831830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e9b69-7062-4963-b4b5-e6b86271b6d3 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:15.838775 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:15.839907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:15.840098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:15.840374 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] PATH_INFO: `/v3/projects/b1d90fc64b7a41fbb59cf5830a988862` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:15.841776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d90fc64b7a41fbb59cf5830a988862)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:15.843873 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:15.892535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d90fc64b7a41fbb59cf5830a988862', 'request_id': 'req-5881cd5f-5cfb-49a4-aca8-755f1a996878'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.901160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d90fc64b7a41fbb59cf5830a988862', 'request_id': 'req-5881cd5f-5cfb-49a4-aca8-755f1a996878'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.907281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d90fc64b7a41fbb59cf5830a988862', 'request_id': 'req-5881cd5f-5cfb-49a4-aca8-755f1a996878'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.918287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] The token cache is being invalidate because project b1d90fc64b7a41fbb59cf5830a988862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:15.918568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.919215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:15.921180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1338/2674] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:15 2022] DELETE /identity/v3/projects/b1d90fc64b7a41fbb59cf5830a988862 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:15.925265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:15.935196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0701076-ef75-48e2-881c-50519ba6a396 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:15.991936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:15.992814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:15.993092 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:15.993363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] PATH_INFO: `/v3/users/9ab6feb90b2a4827bc8a83e425bc425e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:15.994818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab6feb90b2a4827bc8a83e425bc425e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:15.996765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:16.053034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab6feb90b2a4827bc8a83e425bc425e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.066506 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab6feb90b2a4827bc8a83e425bc425e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.067235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab6feb90b2a4827bc8a83e425bc425e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.067868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab6feb90b2a4827bc8a83e425bc425e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.073276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:16.075880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab6feb90b2a4827bc8a83e425bc425e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.092240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1337/2675] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:15 2022] DELETE /identity/v3/users/9ab6feb90b2a4827bc8a83e425bc425e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:39:16.381762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:16.383331 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:16.397184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cd5f-5cfb-49a4-aca8-755f1a996878 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:16.456685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:16.457496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:16.457774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:16.458046 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] PATH_INFO: `/v3/projects/490d75e43bed49cebc61551fdd29559e` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:16.459260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=490d75e43bed49cebc61551fdd29559e)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:16.461155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:16.501300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:16.517126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490d75e43bed49cebc61551fdd29559e', 'request_id': 'req-cabc2314-f38b-421d-ac05-6919297ff464'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.524897 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490d75e43bed49cebc61551fdd29559e', 'request_id': 'req-cabc2314-f38b-421d-ac05-6919297ff464'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.531977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490d75e43bed49cebc61551fdd29559e', 'request_id': 'req-cabc2314-f38b-421d-ac05-6919297ff464'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.543515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] The token cache is being invalidate because project 490d75e43bed49cebc61551fdd29559e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:16.543811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.544536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.546886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1339/2676] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:16 2022] DELETE /identity/v3/projects/490d75e43bed49cebc61551fdd29559e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:16.551232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:16.552915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:16.567016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30318-8da9-428b-9d2d-e7a722daa47c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:16.627235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:16.628134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:16.628392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:16.628666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] PATH_INFO: `/v3/users/c0cc436f99ab49a9a70bd99424516c83` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:16.629900 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0cc436f99ab49a9a70bd99424516c83)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:16.631814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:16.685140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0cc436f99ab49a9a70bd99424516c83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.699617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0cc436f99ab49a9a70bd99424516c83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.700376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0cc436f99ab49a9a70bd99424516c83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.701235 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0cc436f99ab49a9a70bd99424516c83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.709500 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ba671e05-8cbe-4658-91fb-8f6890794d71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0cc436f99ab49a9a70bd99424516c83'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:16.724534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1338/2677] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:16 2022] DELETE /identity/v3/users/c0cc436f99ab49a9a70bd99424516c83 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:17.024463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.025949 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:17.040416 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc2314-f38b-421d-ac05-6919297ff464 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.106098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.106996 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.107279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.107679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] PATH_INFO: `/v3/projects/6135948dec2d4fa78a4e0b854ed00f47` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.109279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6135948dec2d4fa78a4e0b854ed00f47)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.111264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.168566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6135948dec2d4fa78a4e0b854ed00f47', 'request_id': 'req-529ea380-b417-4a9a-93d5-95a07a9eb274'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:17.177702 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6135948dec2d4fa78a4e0b854ed00f47', 'request_id': 'req-529ea380-b417-4a9a-93d5-95a07a9eb274'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:17.183393 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6135948dec2d4fa78a4e0b854ed00f47', 'request_id': 'req-529ea380-b417-4a9a-93d5-95a07a9eb274'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:17.195065 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] The token cache is being invalidate because project 6135948dec2d4fa78a4e0b854ed00f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:17.195366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:17.196048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-529ea380-b417-4a9a-93d5-95a07a9eb274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:17.197649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1340/2678] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:17 2022] DELETE /identity/v3/projects/6135948dec2d4fa78a4e0b854ed00f47 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:17.204615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.204915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.205199 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.228780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:17.230230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:17.304244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1339/2679] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:17.313574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.313863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.314937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.332885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:17.334135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:17.342492 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1341/2680] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:17.346788 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.357246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4c029-b3ce-42fb-a24e-141ac30911d0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.362423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.363097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.363382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.363661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.364404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:17.364715 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.366441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.370318 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:17.386196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1340/2681] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:39:17.388208 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.397790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8336-9d43-4994-8d8b-80be929652c0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.402792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.403443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.403743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.403992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.404704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.406654 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.437619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1342/2682] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:39:17.441738 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.452190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba55f-0174-465c-a438-1f6079895742 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.512945 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.513682 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.513964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.514406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.515123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.517085 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.541220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1341/2683] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:17.544557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.556187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531350e-fb29-48df-9b94-ff1806ceabf5 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.561383 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.562147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.562499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.562782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.563566 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.565177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.576176 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1343/2684] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:39:17.581753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.594572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf01d03-f3dd-4081-ab6b-33186adcee8f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.602541 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d91d4-0783-426d-89d7-9dac8ec4f19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.603598 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d91d4-0783-426d-89d7-9dac8ec4f19d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.604003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d91d4-0783-426d-89d7-9dac8ec4f19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.604267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d91d4-0783-426d-89d7-9dac8ec4f19d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.605212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d91d4-0783-426d-89d7-9dac8ec4f19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.607237 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d91d4-0783-426d-89d7-9dac8ec4f19d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.620910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1342/2685] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:17.626284 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.639493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f3fba-90ce-4b91-bb9e-4c5ba02c4f70 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.644516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.645581 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.645867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.646142 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] PATH_INFO: `/v3/projects/3d9e553f4aa540a08d5983de1b29e6e7/users/edfb9d0a161e4b6399144fa3f0421d76/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.649229 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:17.665531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9e553f4aa540a08d5983de1b29e6e7, user_id=edfb9d0a161e4b6399144fa3f0421d76, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.668195 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.683389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1344/2686] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:17 2022] PUT /identity/v3/projects/3d9e553f4aa540a08d5983de1b29e6e7/users/edfb9d0a161e4b6399144fa3f0421d76/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:17.695354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.695750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.696212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.728140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] MFA Rules not processed for user `edfb9d0a161e4b6399144fa3f0421d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:17.729428 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:17.800168 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1343/2687] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:17.805042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.870105 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c92cd9-b728-4e3c-a9f5-47827972ff4b admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.875467 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.876125 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.876378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.876679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.877340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.879273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:17.909513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1345/2688] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:17.918372 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:17.930553 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365c17e-2a74-45a0-8974-966f60549123 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:17.988406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:17.989077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:17.989365 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:17.989676 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:17.990371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:17.992310 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:18.002120 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:18.016667 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1344/2689] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:17 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:18.021388 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:18.036295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe4f11-75c7-45d2-b1d3-95e9e6542bed admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:18.042835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:18.043506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:18.043827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:18.044109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:18.044846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:18.046931 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:18.062816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1346/2690] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:18.067747 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:18.082933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852970a-f77a-464d-a2c7-0063e987c145 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:18.088081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:18.088969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:18.089244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:18.089513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] PATH_INFO: `/v3/projects/3f331a62c4d14091899c39908496556f/users/b534e5e77d8849fe91232607731efbf5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:18.108094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f331a62c4d14091899c39908496556f, user_id=b534e5e77d8849fe91232607731efbf5, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:18.111369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:18.128843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1345/2691] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:18 2022] PUT /identity/v3/projects/3f331a62c4d14091899c39908496556f/users/b534e5e77d8849fe91232607731efbf5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:18.133742 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:18.145444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a73ebf-34d9-4fb4-a47d-d587342f9c3a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:18.214867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d97042-4438-43d4-b27f-18d15d0f1aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:18.215609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97042-4438-43d4-b27f-18d15d0f1aac admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:18.215997 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97042-4438-43d4-b27f-18d15d0f1aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:18.216369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97042-4438-43d4-b27f-18d15d0f1aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:18.217348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d97042-4438-43d4-b27f-18d15d0f1aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:18.219397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d97042-4438-43d4-b27f-18d15d0f1aac admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:18.230474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1347/2692] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:18.235254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:18.249513 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f40900-81cc-459c-a5c1-6abf30f076d8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:18.255853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:18.256755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:18.257039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:18.257315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] PATH_INFO: `/v3/projects/3f331a62c4d14091899c39908496556f/users/b534e5e77d8849fe91232607731efbf5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:18.259370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f331a62c4d14091899c39908496556f, user_id=b534e5e77d8849fe91232607731efbf5, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:18.262112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:18.277485 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1346/2693] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:18 2022] PUT /identity/v3/projects/3f331a62c4d14091899c39908496556f/users/b534e5e77d8849fe91232607731efbf5/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:18.290245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:18.290580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:18.290853 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:18.320235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] MFA Rules not processed for user `b534e5e77d8849fe91232607731efbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:18.321395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:18.366309 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:18.392540 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1348/2694] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:18.404954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:18.406845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:18.471673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c04a9d-8237-4c76-9b10-4e269f545ca4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:18.476054 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:18.476710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:18.476996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:18.477276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:18.541733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:18.543569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:18.553439 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1347/2695] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:22.838041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:22.839731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:22.853262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb780f-3b48-420b-922b-ea47fef003f7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:22.858052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec874ef-f687-4155-8d93-a109eea9c0d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:22.858700 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec874ef-f687-4155-8d93-a109eea9c0d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:22.858983 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec874ef-f687-4155-8d93-a109eea9c0d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:22.859261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec874ef-f687-4155-8d93-a109eea9c0d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:22.869537 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec874ef-f687-4155-8d93-a109eea9c0d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:22.870977 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec874ef-f687-4155-8d93-a109eea9c0d8 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:22.880911 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1349/2696] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:28.033758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:28.101157 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3113638-0a86-41b3-a717-aa2fce75f4b7 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:28.105869 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:28.106735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:28.107022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:28.107298 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] PATH_INFO: `/v3/users/edfb9d0a161e4b6399144fa3f0421d76` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:28.108531 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfb9d0a161e4b6399144fa3f0421d76)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:28.110576 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:28.125093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:28.179087 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfb9d0a161e4b6399144fa3f0421d76'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.202324 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfb9d0a161e4b6399144fa3f0421d76'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.203137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfb9d0a161e4b6399144fa3f0421d76'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.203814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfb9d0a161e4b6399144fa3f0421d76'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.212968 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfb9d0a161e4b6399144fa3f0421d76'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.232690 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1348/2697] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:28 2022] DELETE /identity/v3/users/edfb9d0a161e4b6399144fa3f0421d76 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:28.237035 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:28.237576 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:28.237661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:28.260251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:28.260251 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:28.325140 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1350/2698] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:28.350210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:28.351775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:28.416391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044fab5-4f9b-449a-8963-78effdcbd541 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:28.419881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:28.420289 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:28.420578 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:28.421198 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:28.431733 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:28.433103 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:28.444423 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1349/2699] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:28.879454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:28.881304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:28.898670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1c049-76ed-49c4-8a7b-829894a3a91b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:28.904881 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:28.906456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:28.907494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:28.907619 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] PATH_INFO: `/v3/projects/3d9e553f4aa540a08d5983de1b29e6e7` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:28.909088 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9e553f4aa540a08d5983de1b29e6e7)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:28.911337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:28.972003 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9e553f4aa540a08d5983de1b29e6e7', 'request_id': 'req-37dfb0f2-9c72-437f-b72c-b7732a18006f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.981087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9e553f4aa540a08d5983de1b29e6e7', 'request_id': 'req-37dfb0f2-9c72-437f-b72c-b7732a18006f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.987850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9e553f4aa540a08d5983de1b29e6e7', 'request_id': 'req-37dfb0f2-9c72-437f-b72c-b7732a18006f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:28.999678 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] The token cache is being invalidate because project 3d9e553f4aa540a08d5983de1b29e6e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:29.000039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.000735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.002603 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1351/2700] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:28 2022] DELETE /identity/v3/projects/3d9e553f4aa540a08d5983de1b29e6e7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:29.006551 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:29.017801 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70043b-b16f-4049-8d77-b732f2dca928 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:29.079547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:29.080434 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.080717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.080995 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] PATH_INFO: `/v3/users/b534e5e77d8849fe91232607731efbf5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.082253 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b534e5e77d8849fe91232607731efbf5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:29.084203 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:29.129609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:29.141261 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b534e5e77d8849fe91232607731efbf5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.153650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b534e5e77d8849fe91232607731efbf5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.154369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b534e5e77d8849fe91232607731efbf5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.155140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b534e5e77d8849fe91232607731efbf5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.163701 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4f5fe078-9e61-4a50-94d3-11b4e0f693fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b534e5e77d8849fe91232607731efbf5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.179830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1350/2701] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:29 2022] DELETE /identity/v3/users/b534e5e77d8849fe91232607731efbf5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:29.479988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:29.481939 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:29.505008 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfb0f2-9c72-437f-b72c-b7732a18006f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:29.556279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:29.568319 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:29.569392 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.569798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.570122 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] PATH_INFO: `/v3/projects/3f331a62c4d14091899c39908496556f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.571658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f331a62c4d14091899c39908496556f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:29.573758 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:29.623671 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f331a62c4d14091899c39908496556f', 'request_id': 'req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.632264 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f331a62c4d14091899c39908496556f', 'request_id': 'req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.638803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f331a62c4d14091899c39908496556f', 'request_id': 'req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.650739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] The token cache is being invalidate because project 3f331a62c4d14091899c39908496556f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:29.651028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.651704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-8e42e098-d349-4bb3-a1e6-4d8d18ff101d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:29.653300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1352/2702] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:29 2022] DELETE /identity/v3/projects/3f331a62c4d14091899c39908496556f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:29.659532 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.659911 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.660196 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.680093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:29.681346 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:29.743283 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1351/2703] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:29.752300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.752588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.752866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.772375 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:29.773563 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:29.782985 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1353/2704] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:29.787953 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:29.801666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416302-ddf9-4b16-9db1-9a8aa80a29a4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:29.808243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:29.808947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.809229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.809534 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.810262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:29.810571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:29.812655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:29.825620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1352/2705] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:29.830398 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:29.840095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e433198-17e6-4d70-a0cf-c77f948fab7c None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:29.845765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:29.846447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.846733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.847007 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.847704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:29.849691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:29.877642 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1354/2706] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:29 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:29.881697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:29.891872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50775fe-6101-4d57-aac2-d7e6f2cd628e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:29.952868 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:29.953564 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:29.953845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:29.954116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:29.954818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:29.956728 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:29.978744 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1353/2707] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:29 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:29.983146 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:29.994064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec7c97-e1c9-4426-96ea-06731c9cb337 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.000080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.000765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.001052 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.001328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.002095 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.003761 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.014849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1355/2708] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:39:30.020276 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.034661 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388c0f3-d349-4c82-9352-e526f424cf14 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.043370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606958b-fdc4-45cf-8c32-7bd902e2e97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.044280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606958b-fdc4-45cf-8c32-7bd902e2e97c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.044673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606958b-fdc4-45cf-8c32-7bd902e2e97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.045069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606958b-fdc4-45cf-8c32-7bd902e2e97c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.046273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606958b-fdc4-45cf-8c32-7bd902e2e97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.047871 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606958b-fdc4-45cf-8c32-7bd902e2e97c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.058154 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1354/2709] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:30.062351 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.072481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2005-4c2c-4dc7-9c3a-2b99a4e85be1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.078475 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.079366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.079630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.079903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] PATH_INFO: `/v3/projects/b80c731b0d6f45b7b2f571de98a890c5/users/dc70e3cb874f4b2bb451b08f1d93597f/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.096444 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80c731b0d6f45b7b2f571de98a890c5, user_id=dc70e3cb874f4b2bb451b08f1d93597f, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.100755 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.120110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1356/2710] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:30 2022] PUT /identity/v3/projects/b80c731b0d6f45b7b2f571de98a890c5/users/dc70e3cb874f4b2bb451b08f1d93597f/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:39:30.132746 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.133249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.133384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.168983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] MFA Rules not processed for user `dc70e3cb874f4b2bb451b08f1d93597f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:30.170669 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:30.179922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:30.249117 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1355/2711] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:30.253820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.320568 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14055c07-63eb-4be2-8f0e-0c8b2b35f9e6 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.325717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.326367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.326665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.326946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.327628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.329543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.356672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1357/2712] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:30 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:30.361519 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.375229 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3902ff-daf1-4103-91d6-e138ea441ed4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.441096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.441807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.442093 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.442370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.443065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.445023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.465896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1356/2713] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:30 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:30.470432 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.476267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:30.483796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f295b-cee6-4e21-b01e-f40127a3cb31 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.490061 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.490940 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.491220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.491493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.492231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.493874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.504799 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1358/2714] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:39:30.509375 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.520707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e6f1a-ad4b-41bc-85e2-1fe7684fe5c0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.525847 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.526751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.527034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.527311 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] PATH_INFO: `/v3/projects/3708d461da81492eb6eea24ebbfa7ddf/users/8aa75e2e0b02468da55c9ff25cb758e2/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.544334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3708d461da81492eb6eea24ebbfa7ddf, user_id=8aa75e2e0b02468da55c9ff25cb758e2, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.547693 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.561899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1357/2715] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:30 2022] PUT /identity/v3/projects/3708d461da81492eb6eea24ebbfa7ddf/users/8aa75e2e0b02468da55c9ff25cb758e2/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:30.565888 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.576260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c699-dd5f-4475-ace3-6569c6f81a16 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.640012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9f4d0-6481-4435-b47f-4b641cd870f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.640674 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9f4d0-6481-4435-b47f-4b641cd870f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.640948 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9f4d0-6481-4435-b47f-4b641cd870f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.641256 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9f4d0-6481-4435-b47f-4b641cd870f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.642005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9f4d0-6481-4435-b47f-4b641cd870f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.643597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9f4d0-6481-4435-b47f-4b641cd870f8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.653315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1359/2716] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:30.659401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.674165 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57359257-947e-46e9-b1f8-1f0f7fd38d63 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.679511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.680410 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.680697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.680975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] PATH_INFO: `/v3/projects/3708d461da81492eb6eea24ebbfa7ddf/users/8aa75e2e0b02468da55c9ff25cb758e2/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.683096 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3708d461da81492eb6eea24ebbfa7ddf, user_id=8aa75e2e0b02468da55c9ff25cb758e2, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:30.685925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:30.696191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1358/2717] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:30 2022] PUT /identity/v3/projects/3708d461da81492eb6eea24ebbfa7ddf/users/8aa75e2e0b02468da55c9ff25cb758e2/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:30.762214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.762477 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.762913 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.794002 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] MFA Rules not processed for user `8aa75e2e0b02468da55c9ff25cb758e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:30.795205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:30.878796 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1360/2718] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:30.890243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:30.892223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:30.955068 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cc99e-f9f7-4958-85e0-e9a5b5fa102a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:30.959460 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569954-6056-4315-92ca-f3dd74e1a788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:30.960107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569954-6056-4315-92ca-f3dd74e1a788 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:30.960393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569954-6056-4315-92ca-f3dd74e1a788 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:30.960674 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569954-6056-4315-92ca-f3dd74e1a788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:30.973648 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:31.021814 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20569954-6056-4315-92ca-f3dd74e1a788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:31.023279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20569954-6056-4315-92ca-f3dd74e1a788 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:31.032608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1359/2719] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:33.597053 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:33.665500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07212be7-75fa-4a2b-b80a-c2dc148558b0 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:33.670446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:33.671801 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:33.672077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:33.672385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] PATH_INFO: `/v3/users/dc70e3cb874f4b2bb451b08f1d93597f` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:33.673625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc70e3cb874f4b2bb451b08f1d93597f)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:33.675531 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:33.689255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:33.725586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc70e3cb874f4b2bb451b08f1d93597f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:33.734579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc70e3cb874f4b2bb451b08f1d93597f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:33.735182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc70e3cb874f4b2bb451b08f1d93597f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:33.735837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc70e3cb874f4b2bb451b08f1d93597f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:33.747367 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc70e3cb874f4b2bb451b08f1d93597f'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:33.760337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1361/2720] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:33 2022] DELETE /identity/v3/users/dc70e3cb874f4b2bb451b08f1d93597f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:33.765150 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:33.765697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:33.765697 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:33.783670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:33.784908 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:33.865344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1360/2721] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:33.877114 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:33.878750 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:33.941344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b224e-5532-4dda-86dc-a7fa282eec73 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:33.945364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:33.946020 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:33.946299 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:33.946569 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:33.955835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:33.957261 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:33.966630 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1362/2722] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:34.259530 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:34.261349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:34.279488 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfec37e-87d9-4d4f-9c76-9399a0593e5f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:34.284824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:34.285725 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:34.286007 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:34.286287 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] PATH_INFO: `/v3/projects/b80c731b0d6f45b7b2f571de98a890c5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:34.287545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80c731b0d6f45b7b2f571de98a890c5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:34.289717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:34.340910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80c731b0d6f45b7b2f571de98a890c5', 'request_id': 'req-271b4030-ab6d-46fc-9635-0199f138ae0d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.349615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80c731b0d6f45b7b2f571de98a890c5', 'request_id': 'req-271b4030-ab6d-46fc-9635-0199f138ae0d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.356264 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80c731b0d6f45b7b2f571de98a890c5', 'request_id': 'req-271b4030-ab6d-46fc-9635-0199f138ae0d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.367116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] The token cache is being invalidate because project b80c731b0d6f45b7b2f571de98a890c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:34.367344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.368052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.369659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1361/2723] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:34 2022] DELETE /identity/v3/projects/b80c731b0d6f45b7b2f571de98a890c5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:34.374019 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:34.386001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5518c23-d1f7-4487-8a6e-e23df0ffa44e service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:34.452217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:34.452995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:34.453266 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:34.453649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] PATH_INFO: `/v3/users/8aa75e2e0b02468da55c9ff25cb758e2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:34.454837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa75e2e0b02468da55c9ff25cb758e2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:34.456789 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:34.495946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:34.507785 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa75e2e0b02468da55c9ff25cb758e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.515652 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa75e2e0b02468da55c9ff25cb758e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.516306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa75e2e0b02468da55c9ff25cb758e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.516927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa75e2e0b02468da55c9ff25cb758e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.531081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-38945497-8ff1-4a7e-9f69-f62f4acf9114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa75e2e0b02468da55c9ff25cb758e2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:34.547028 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1363/2724] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:39:34 2022] DELETE /identity/v3/users/8aa75e2e0b02468da55c9ff25cb758e2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:34.891600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:34.893175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:34.909729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b4030-ab6d-46fc-9635-0199f138ae0d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:34.946309 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:34.971647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:34.972511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:34.972787 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:34.973059 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] PATH_INFO: `/v3/projects/3708d461da81492eb6eea24ebbfa7ddf` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:34.974255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3708d461da81492eb6eea24ebbfa7ddf)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:34.976212 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.028563 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3708d461da81492eb6eea24ebbfa7ddf', 'request_id': 'req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:35.037559 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3708d461da81492eb6eea24ebbfa7ddf', 'request_id': 'req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:35.046981 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3708d461da81492eb6eea24ebbfa7ddf', 'request_id': 'req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:35.062967 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] The token cache is being invalidate because project 3708d461da81492eb6eea24ebbfa7ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:39:35.063254 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:35.063983 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a8ecf0ee-02a4-4104-8cfc-5e07d37c8d14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:39:35.065635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1362/2725] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:39:34 2022] DELETE /identity/v3/projects/3708d461da81492eb6eea24ebbfa7ddf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:39:35.071082 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.071606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.071606 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.090846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:35.092045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:35.152866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1364/2726] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:39:35.160664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.160949 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.161240 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.180876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:35.182164 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:35.191255 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1363/2727] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:35.195718 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.206675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bddb1f-b179-4500-8e02-10dd3fd79970 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.213716 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.214828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.215292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.215766 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.217064 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:39:35.217701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.220594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.234765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1365/2728] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:39:35.239382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.249102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730266de-18d0-4d26-bfcb-e2d3a09f5b4b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.254282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.254939 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.255222 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.255499 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.256219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.258181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.287072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1364/2729] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:35.291947 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.303641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59edad-9c6f-4be4-a67f-58207c08d4d2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.365404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.366097 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.366382 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.366658 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.367342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.369267 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.392363 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1366/2730] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:35.396562 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.407181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e5bc-f426-48a1-96ac-4671c409e219 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.412465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.413122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.413465 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.413749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.414481 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.416097 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.427138 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1365/2731] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:35.431665 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.445275 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7423ec-8d33-44cd-b2a6-36c7ad8fc41f admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.451757 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd95d8-62a7-450c-95b8-835b96a0f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.452587 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd95d8-62a7-450c-95b8-835b96a0f50d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.452784 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd95d8-62a7-450c-95b8-835b96a0f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.453054 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd95d8-62a7-450c-95b8-835b96a0f50d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.453791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd95d8-62a7-450c-95b8-835b96a0f50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.455481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd95d8-62a7-450c-95b8-835b96a0f50d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.465866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1367/2732] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:35.470267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.480505 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18853d05-7ff1-461a-aa65-b696793d6be4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.485833 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.486751 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.487027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.487301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] PATH_INFO: `/v3/projects/430899def10d40c192f0d54b8ebf0af5/users/9485d40bf82e421bafbb714c60c96df2/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.502835 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430899def10d40c192f0d54b8ebf0af5, user_id=9485d40bf82e421bafbb714c60c96df2, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.505542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.521813 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1366/2733] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:35 2022] PUT /identity/v3/projects/430899def10d40c192f0d54b8ebf0af5/users/9485d40bf82e421bafbb714c60c96df2/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:35.527123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03e85-8150-4267-89f3-3df1ec6f77ff None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.527412 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03e85-8150-4267-89f3-3df1ec6f77ff None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.527695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c03e85-8150-4267-89f3-3df1ec6f77ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.547022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-54c03e85-8150-4267-89f3-3df1ec6f77ff None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:35.548262 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-54c03e85-8150-4267-89f3-3df1ec6f77ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:35.561972 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:35.619502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1368/2734] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:35.630288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:35.632194 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:35.701391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc61b-f5b2-4ed7-b60b-e804e308f0f8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:35.706432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:35.707083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:35.707363 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:35.707673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:35.718388 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:35.719853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:35.729727 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1367/2735] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:40.510782 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.511285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.511495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.543820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] MFA Rules not processed for user `9485d40bf82e421bafbb714c60c96df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:40.545344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:40.619072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1369/2736] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:40.623843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.633894 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c07fd5-4cdc-46ee-b0db-c08d95804f4e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.639012 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.639858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.640391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.640688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.641429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.643430 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.647191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:40.675236 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1368/2737] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:39:40.679334 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.690621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a61353-17e4-4ad0-a76c-56b44e5781be None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.746086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.746816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.747042 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.747317 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.748021 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.750083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.773056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1370/2738] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:39:40 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:39:40.777349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.788526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5dbccc-fcc3-45b0-a376-b3208af75dea admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.793768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.794427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.794713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.794988 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.796249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.797902 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.808982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1369/2739] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:39:40.813231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.823274 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a35d92-e867-4131-b4ff-edd3298aeca1 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.828215 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.829203 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.829612 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.829893 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] PATH_INFO: `/v3/projects/d869c798457348fdbf3118494dd5775e/users/3d9e25d300f0443a84d122964bfae1c4/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.845165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d869c798457348fdbf3118494dd5775e, user_id=3d9e25d300f0443a84d122964bfae1c4, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.847908 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.867360 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1371/2740] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:40 2022] PUT /identity/v3/projects/d869c798457348fdbf3118494dd5775e/users/3d9e25d300f0443a84d122964bfae1c4/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:40.868369 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.879904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b0dc5-3a26-4a25-b704-32e826adc574 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.944708 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ea019-35fd-449e-8924-4205f4a074a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.945780 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea019-35fd-449e-8924-4205f4a074a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.946183 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea019-35fd-449e-8924-4205f4a074a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.946431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea019-35fd-449e-8924-4205f4a074a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.947539 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea019-35fd-449e-8924-4205f4a074a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.949980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea019-35fd-449e-8924-4205f4a074a6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:40.961249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1370/2741] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:39:40.965736 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:40.978641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9526bf-90bc-4b35-afb4-e5aae88ca9a8 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:40.984177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:40.985060 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:40.985493 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:40.985772 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] PATH_INFO: `/v3/projects/d869c798457348fdbf3118494dd5775e/users/3d9e25d300f0443a84d122964bfae1c4/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:40.988107 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d869c798457348fdbf3118494dd5775e, user_id=3d9e25d300f0443a84d122964bfae1c4, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:40.991127 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:41.003594 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1372/2742] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:39:40 2022] PUT /identity/v3/projects/d869c798457348fdbf3118494dd5775e/users/3d9e25d300f0443a84d122964bfae1c4/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:39:42.685143 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9cd19-6c17-4faf-a789-fbe78c8987ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:42.685683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9cd19-6c17-4faf-a789-fbe78c8987ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:42.686175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9cd19-6c17-4faf-a789-fbe78c8987ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:42.716791 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ebf9cd19-6c17-4faf-a789-fbe78c8987ac None None] MFA Rules not processed for user `6883da006d3f4aec8805c74cf5ffae86`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:42.718782 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ebf9cd19-6c17-4faf-a789-fbe78c8987ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:42.804545 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1371/2743] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:43.294246 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:43.334228 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:43.501273 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9584c-3fa9-44c9-826d-e200b13b6e47 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:43.515542 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:43.516516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:43.516802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:43.517083 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:43.527743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:43.557850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:43.559287 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:43.580455 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1373/2744] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:45.974270 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522cf2-608b-4ccc-b8bd-ce8ca64514fa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:45.974702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522cf2-608b-4ccc-b8bd-ce8ca64514fa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:45.974929 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522cf2-608b-4ccc-b8bd-ce8ca64514fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:46.002824 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b2522cf2-608b-4ccc-b8bd-ce8ca64514fa None None] MFA Rules not processed for user `3d9e25d300f0443a84d122964bfae1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:46.004334 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2522cf2-608b-4ccc-b8bd-ce8ca64514fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:46.072683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:46.078122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1372/2745] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:46.092574 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:39:46.094178 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:46.214281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89c7e-8d3a-4861-93b3-e1a5527de9cc service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:39:46.220802 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:39:46.221518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:46.221804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:46.222086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:46.307138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:39:46.308902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:39:46.328066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1374/2746] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:39:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:39:47.359286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:39:47.360259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:39:47.360865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:39:47.392950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] MFA Rules not processed for user `9485d40bf82e421bafbb714c60c96df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:39:47.394991 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:39:47.407416 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1373/2747] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:39:56.315992 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=81 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:39:57.386142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:03.979727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:03.981290 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:04.054316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8381b13-9c82-4498-8c3c-da7ada5cadc5 service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:04.060110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:04.060781 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:04.061078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:04.061354 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:04.063295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:40:04.075514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:04.077134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:04.087685 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1375/2748] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:40:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:40:14.083385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=32 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:40:20.384844 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:40:20.395637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99561436-ee10-4ed7-9202-3b1476c969ab None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:40:20.400171 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:40:20.400887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:40:20.401172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:40:20.401517 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:40:20.412489 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:40:20.413998 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:40:20.426415 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1374/2749] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:40:20 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:40:30.419745 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:09.193636 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.195308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:09.264423 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553e964-a59f-43ef-a1a0-5ec0ebf504c4 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.269930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.270848 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.271128 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.271402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] PATH_INFO: `/v3/users/6883da006d3f4aec8805c74cf5ffae86` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.272655 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6883da006d3f4aec8805c74cf5ffae86)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.274903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.331935 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6883da006d3f4aec8805c74cf5ffae86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.340919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6883da006d3f4aec8805c74cf5ffae86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.341629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6883da006d3f4aec8805c74cf5ffae86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.342247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6883da006d3f4aec8805c74cf5ffae86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.355071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4d817f15-7311-4fee-ac03-1fe5bf816d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6883da006d3f4aec8805c74cf5ffae86'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.370010 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1376/2750] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:41:09 2022] DELETE /identity/v3/users/6883da006d3f4aec8805c74cf5ffae86 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:09.561444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.579234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c70179-b78f-4f72-8686-2f7ac557a13e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.645638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.646520 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.646800 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.647070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] PATH_INFO: `/v3/projects/b1222038740244f69a977fe658d9bfe0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.648376 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1222038740244f69a977fe658d9bfe0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.650354 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.703614 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1222038740244f69a977fe658d9bfe0', 'request_id': 'req-1fa8b4f7-84de-4379-9f55-fcb625ff7060'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.711887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1222038740244f69a977fe658d9bfe0', 'request_id': 'req-1fa8b4f7-84de-4379-9f55-fcb625ff7060'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.718958 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1222038740244f69a977fe658d9bfe0', 'request_id': 'req-1fa8b4f7-84de-4379-9f55-fcb625ff7060'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.736859 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] The token cache is being invalidate because project b1222038740244f69a977fe658d9bfe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:41:09.737144 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.737984 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1fa8b4f7-84de-4379-9f55-fcb625ff7060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:41:09.739671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1375/2751] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:41:09 2022] DELETE /identity/v3/projects/b1222038740244f69a977fe658d9bfe0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:09.744952 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.745249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.745547 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.765161 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:09.766662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:09.833182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1377/2752] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:41:09.841298 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.841617 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.841899 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.860946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:09.862223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:09.870634 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1376/2753] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:41:09.875090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.885356 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f120e-6afd-4437-a927-f4396b51d53a None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.890449 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.891109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.891440 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.891651 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.892387 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:41:09.892701 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.894887 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.910366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1378/2754] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 13:41:09.914946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.925105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad6c18-7afd-4078-af64-1e6628e281f8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.930569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:09.931224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:09.931511 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:09.931819 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:09.932516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:09.934484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:09.966317 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1377/2755] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:41:09 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:09.972304 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:09.982589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d909f8-59c2-4763-8134-d0f4e5798d0c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:09.986311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.043495 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.044187 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.044812 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.045086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.045847 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.047770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.072306 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1379/2756] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:41:09 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:41:10.077971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.090381 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691aae1-90ee-4d35-bcd9-87e3ad5632d6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.096718 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.098015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.098301 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.098580 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.099305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.100970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.113021 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1378/2757] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:10.118545 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.128925 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c21e01-ca67-4c19-957a-588e4ddcbdc4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.134793 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a666e62-f51b-43bb-8a05-95bb496164ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.135458 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a666e62-f51b-43bb-8a05-95bb496164ca admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.135734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a666e62-f51b-43bb-8a05-95bb496164ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.136005 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a666e62-f51b-43bb-8a05-95bb496164ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.136735 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a666e62-f51b-43bb-8a05-95bb496164ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.138361 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a666e62-f51b-43bb-8a05-95bb496164ca admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.149086 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1380/2758] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:41:10.155285 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.166686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed3aad-caa2-4393-aa5b-5cda5abc8500 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.172831 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.173758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.174011 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.174280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] PATH_INFO: `/v3/projects/bc560e8b75194743bbb46d903b481357/users/daa4a2a39deb4e7ba54032acec14e029/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.189393 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc560e8b75194743bbb46d903b481357, user_id=daa4a2a39deb4e7ba54032acec14e029, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.192109 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.206637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1379/2759] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:41:10 2022] PUT /identity/v3/projects/bc560e8b75194743bbb46d903b481357/users/daa4a2a39deb4e7ba54032acec14e029/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:41:10.212166 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ae0c-8358-4d65-83d9-fb891ba674dc None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.212695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ae0c-8358-4d65-83d9-fb891ba674dc None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.212695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32ae0c-8358-4d65-83d9-fb891ba674dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.230807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-bc32ae0c-8358-4d65-83d9-fb891ba674dc None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:10.232000 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-bc32ae0c-8358-4d65-83d9-fb891ba674dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:10.293118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1381/2760] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:41:10.304039 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:10.306107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:10.339230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:10.376259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21252ae4-00dd-40a2-9cdd-25c664b64faa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:10.380881 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4388b83-25f4-42c9-8391-d70d3d83e627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:10.381594 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4388b83-25f4-42c9-8391-d70d3d83e627 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:10.381880 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4388b83-25f4-42c9-8391-d70d3d83e627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:10.382195 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4388b83-25f4-42c9-8391-d70d3d83e627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:10.392455 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4388b83-25f4-42c9-8391-d70d3d83e627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:10.394013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4388b83-25f4-42c9-8391-d70d3d83e627 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:10.404023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1380/2761] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:41:14.285067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.285589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.285710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.305646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] MFA Rules not processed for user `aa0a4b41707d444581ee0aa2296ac0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:14.306989 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:14.417984 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1382/2762] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:41:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:41:14.859482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedc15e-b334-49c8-bc0d-e1992e8b534c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:14.860295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedc15e-b334-49c8-bc0d-e1992e8b534c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:14.860571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedc15e-b334-49c8-bc0d-e1992e8b534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:14.893699 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1eedc15e-b334-49c8-bc0d-e1992e8b534c None None] MFA Rules not processed for user `daa4a2a39deb4e7ba54032acec14e029`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:14.895174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1eedc15e-b334-49c8-bc0d-e1992e8b534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:14.981659 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1381/2763] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:41:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:41:15.001497 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:15.003055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:15.045616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:15.086348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397b172-9f08-4125-aad6-edadd0e5a28e None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:15.092017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:15.092673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:15.092923 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:15.093220 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:15.101500 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:15.103110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:15.112295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1383/2764] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:41:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:41:16.553547 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:16.554064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:16.554064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:16.582010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] MFA Rules not processed for user `daa4a2a39deb4e7ba54032acec14e029`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:41:16.583586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:16.592970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1382/2765] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:41:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:41:24.795519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:24.805898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3d903-3f6a-48a5-86cc-81271e3d5acb service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:24.810631 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:24.811308 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:24.811770 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:24.812085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:24.814036 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:24.829681 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:24.831190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:24.842643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1384/2766] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:41:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:41:26.571239 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=49 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:27.366208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:27.367933 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:27.380159 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2044d1-33f3-4f98-9a6f-2e299bbe9591 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:27.384395 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:27.385095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:27.385362 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:27.385672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:27.387588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:27.403297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:27.404770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c9fde-6f6c-400e-9f83-7cebdce5c8ea service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:27.417360 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1383/2767] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:41:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:41:31.037753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:41:31.050436 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de7118-d59b-47e9-bfb9-7027d8ea61c1 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:41:31.058167 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:41:31.058820 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:41:31.059098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:41:31.059377 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:41:31.061645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:41:31.073378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:41:31.074995 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:41:31.084474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1385/2768] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:41:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:41:37.412478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:41:41.080024 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=32 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:15.571781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:15.572405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:15.572405 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:15.591243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] MFA Rules not processed for user `daa4a2a39deb4e7ba54032acec14e029`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:42:15.592552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:15.600111 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1384/2769] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:42:16.243307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:16.245397 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:16.323059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3eca-9c72-4fbb-a754-90b0de677b5a service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:16.328405 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:16.329051 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:16.329330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:16.329635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:16.331611 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:16.343880 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:16.345461 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:16.355348 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1386/2770] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:42:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:42:18.426677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:18.437294 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03abe3-3f84-4629-ac8b-957c37759b0d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:18.440875 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:18.441568 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:18.441947 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:18.442300 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:18.444082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:18.455035 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:18.456524 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:18.469479 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1385/2771] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:42:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:42:19.156794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:19.176385 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e136a-f311-446e-96b6-ae353c639a72 service placement] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:19.182683 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:19.183404 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:19.183684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:19.184145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:19.186386 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:19.204321 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:19.205990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:19.217845 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1387/2772] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:42:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:42:22.893913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:22.909660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509f21d-725f-4985-8306-5adf3a7b422c service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:22.916079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:22.916807 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:22.917201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:22.917552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:22.920063 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:22.934292 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:22.936062 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:22.948366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1386/2773] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:42:29.017816 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:29.027616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fe905-f5e8-47a8-a40d-6bbf02a6b33e service nova] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.032040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.032693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.032998 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.033257 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.035420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:29.047094 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.048560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.057184 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1388/2774] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:42:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:42:29.157886 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:42:29.168141 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a196b7c-35b9-4f5a-9b77-7b93645c25c7 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:42:29.172926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:42:29.173618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:42:29.173897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:42:29.174172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:42:29.176077 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:42:29.191611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:42:29.193110 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:42:29.204974 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1387/2775] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:42:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:42:39.053320 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=44 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:42:39.199714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=21 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:06.211837 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:06.213470 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:06.230657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117f8a6-eedf-4ec8-803f-b24aa8c9de51 service neutron] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:06.238038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '862f9782f8574b8bb99d38c51333277f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:06.238688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:06.238973 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:06.239244 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:06.241074 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:06.258081 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:06.259714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85871ae3-ac67-4a03-b730-f4150b0c54f1 service nova] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:06.271805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1389/2776] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:43:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:43:16.266621 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:17.441079 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:17.462491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0322e-fb14-47e1-b9ce-4a63e4395d85 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:17.466295 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:17.467042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:17.467409 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:17.467695 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:17.469542 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:17.536016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:17.537632 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:17.562286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1388/2777] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:43:18.545667 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:18.546625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:18.546912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:18.566903 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] MFA Rules not processed for user `daa4a2a39deb4e7ba54032acec14e029`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:43:18.572896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:18.582649 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1390/2778] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:43:19.368673 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:19.397710 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6d266-8adc-4267-983a-ffccc2f7b80e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:19.401686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:19.402328 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:19.402613 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:19.402889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:19.404997 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:19.428555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:19.430114 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:19.438825 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1389/2779] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:43:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:43:24.940715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:24.942439 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:24.961790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfd23-c3dc-4bff-9252-e29d09b1cfc9 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:24.967148 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:24.968827 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:24.969113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:24.969395 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] PATH_INFO: `/v3/users/aa0a4b41707d444581ee0aa2296ac0ba` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:24.970732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0a4b41707d444581ee0aa2296ac0ba)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:24.973009 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:25.025546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0a4b41707d444581ee0aa2296ac0ba'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.034206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0a4b41707d444581ee0aa2296ac0ba'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.034806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0a4b41707d444581ee0aa2296ac0ba'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.035413 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0a4b41707d444581ee0aa2296ac0ba'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.048016 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0a4b41707d444581ee0aa2296ac0ba'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.059951 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1391/2780] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:43:24 2022] DELETE /identity/v3/users/aa0a4b41707d444581ee0aa2296ac0ba => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:43:25.254493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:25.271302 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0a463-3537-42d2-917e-2cd13a8b71a9 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:25.331838 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:25.332750 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:25.333069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:25.333347 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] PATH_INFO: `/v3/projects/98ad7b921dbc4a8d9e9be0d50385deee` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:25.334608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98ad7b921dbc4a8d9e9be0d50385deee)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:25.336595 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:25.382969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:25.390536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98ad7b921dbc4a8d9e9be0d50385deee', 'request_id': 'req-671120f0-02d1-42e5-a96e-6fce6c42d056'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.398552 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98ad7b921dbc4a8d9e9be0d50385deee', 'request_id': 'req-671120f0-02d1-42e5-a96e-6fce6c42d056'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.405495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98ad7b921dbc4a8d9e9be0d50385deee', 'request_id': 'req-671120f0-02d1-42e5-a96e-6fce6c42d056'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.416015 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] The token cache is being invalidate because project 98ad7b921dbc4a8d9e9be0d50385deee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:43:25.416242 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.416863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.418778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1390/2781] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:43:25 2022] DELETE /identity/v3/projects/98ad7b921dbc4a8d9e9be0d50385deee => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:43:25.423171 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:25.424722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:25.438494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f230f6-e340-4283-b9f1-b79d65eaf879 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:25.507670 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:25.508560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:25.508841 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:25.509111 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] PATH_INFO: `/v3/users/e9ed9b0406de40df8843e3e137a1303d` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:25.510312 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ed9b0406de40df8843e3e137a1303d)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:25.512227 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:25.565442 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ed9b0406de40df8843e3e137a1303d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.573840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ed9b0406de40df8843e3e137a1303d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.574483 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ed9b0406de40df8843e3e137a1303d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.575087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ed9b0406de40df8843e3e137a1303d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.585590 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:25.589593 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ed9b0406de40df8843e3e137a1303d'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.605734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1392/2782] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:43:25 2022] DELETE /identity/v3/users/e9ed9b0406de40df8843e3e137a1303d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:43:25.787067 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:43:25.788628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:43:25.802794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671120f0-02d1-42e5-a96e-6fce6c42d056 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:43:25.877025 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:43:25.877963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:43:25.878247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:43:25.878523 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] PATH_INFO: `/v3/projects/1564d5d1d11f43fa9abc64a0fe12818a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:43:25.886350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1564d5d1d11f43fa9abc64a0fe12818a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:43:25.888399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:43:25.947688 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1564d5d1d11f43fa9abc64a0fe12818a', 'request_id': 'req-aee02753-0dc5-43cf-9b0d-c051a0527a70'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.958206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1564d5d1d11f43fa9abc64a0fe12818a', 'request_id': 'req-aee02753-0dc5-43cf-9b0d-c051a0527a70'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.969387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1564d5d1d11f43fa9abc64a0fe12818a', 'request_id': 'req-aee02753-0dc5-43cf-9b0d-c051a0527a70'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.981670 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] The token cache is being invalidate because project 1564d5d1d11f43fa9abc64a0fe12818a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:43:25.981964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.982763 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:43:25.985140 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1391/2783] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:43:25 2022] DELETE /identity/v3/projects/1564d5d1d11f43fa9abc64a0fe12818a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:43:35.602272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=1,keystone:SELECT=6 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:43:35.978338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=60,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:07.512518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:07.514056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:07.528945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca92f5f-ff20-4302-bfb4-5f0affdcb2a7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:07.592443 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:07.593344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:07.593680 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:07.593950 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] PATH_INFO: `/v3/users/9485d40bf82e421bafbb714c60c96df2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:07.595192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9485d40bf82e421bafbb714c60c96df2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:07.597180 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:07.652994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9485d40bf82e421bafbb714c60c96df2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:07.661522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9485d40bf82e421bafbb714c60c96df2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:07.662152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9485d40bf82e421bafbb714c60c96df2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:07.662765 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9485d40bf82e421bafbb714c60c96df2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:07.675450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9485d40bf82e421bafbb714c60c96df2'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:07.687570 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1393/2784] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:44:07 2022] DELETE /identity/v3/users/9485d40bf82e421bafbb714c60c96df2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:44:07.872394 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:07.873971 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:07.892770 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee02753-0dc5-43cf-9b0d-c051a0527a70 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:07.960978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:07.961904 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:07.962191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:07.963304 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] PATH_INFO: `/v3/projects/430899def10d40c192f0d54b8ebf0af5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:07.963675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430899def10d40c192f0d54b8ebf0af5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:07.965647 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:08.018046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430899def10d40c192f0d54b8ebf0af5', 'request_id': 'req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.028306 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430899def10d40c192f0d54b8ebf0af5', 'request_id': 'req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.037013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430899def10d40c192f0d54b8ebf0af5', 'request_id': 'req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.048726 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] The token cache is being invalidate because project 430899def10d40c192f0d54b8ebf0af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:44:08.049013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.049706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.051687 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1392/2785] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:44:07 2022] DELETE /identity/v3/projects/430899def10d40c192f0d54b8ebf0af5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:44:08.055819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:08.057369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:08.070368 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0f626-5fb4-43fe-890c-272bf7480f6e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:08.127920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:08.128863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:08.129143 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:08.129445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] PATH_INFO: `/v3/users/3d9e25d300f0443a84d122964bfae1c4` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:08.130698 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d9e25d300f0443a84d122964bfae1c4)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:08.133149 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:08.154956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:08.183745 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d9e25d300f0443a84d122964bfae1c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.192055 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d9e25d300f0443a84d122964bfae1c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.192714 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d9e25d300f0443a84d122964bfae1c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.193345 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d9e25d300f0443a84d122964bfae1c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.205791 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-82412aef-1c24-4b4e-919d-3756bf70b179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d9e25d300f0443a84d122964bfae1c4'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.218011 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1394/2786] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:44:08 2022] DELETE /identity/v3/users/3d9e25d300f0443a84d122964bfae1c4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:44:08.401475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:44:08.403053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:08.420321 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9b21-0751-47af-8762-ce9cf8bb2ee3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:44:08.479516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:44:08.480336 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:08.480619 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:08.480897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] PATH_INFO: `/v3/projects/d869c798457348fdbf3118494dd5775e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:08.482149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d869c798457348fdbf3118494dd5775e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:44:08.484094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:44:08.520822 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:08.534834 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd869c798457348fdbf3118494dd5775e', 'request_id': 'req-eb93d0f3-d46e-44ad-be09-b2808fa64912'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.544451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd869c798457348fdbf3118494dd5775e', 'request_id': 'req-eb93d0f3-d46e-44ad-be09-b2808fa64912'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.551689 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd869c798457348fdbf3118494dd5775e', 'request_id': 'req-eb93d0f3-d46e-44ad-be09-b2808fa64912'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.563186 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] The token cache is being invalidate because project d869c798457348fdbf3118494dd5775e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:44:08.563471 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.564189 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:44:08.565878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1393/2787] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:44:08 2022] DELETE /identity/v3/projects/d869c798457348fdbf3118494dd5775e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:44:18.214379 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:18.559735 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:44:21.376656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:44:21.377190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:44:21.377190 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:44:21.395909 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] MFA Rules not processed for user `daa4a2a39deb4e7ba54032acec14e029`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:44:21.397173 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:44:21.456340 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1395/2788] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:44:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:44:31.448501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=25 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:21.453288 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:21.454779 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:21.560137 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93d0f3-d46e-44ad-be09-b2808fa64912 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:21.565314 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:21.566241 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:21.566526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:21.566805 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:21.568703 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:21.581810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:21.583494 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:21.593952 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1394/2789] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:45:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:29.870807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:29.872310 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:29.954156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745c911-6ccb-4776-8f55-3dbfd08dca98 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:29.958840 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:29.959726 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:29.960012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:29.960288 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] PATH_INFO: `/v3/users/daa4a2a39deb4e7ba54032acec14e029` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:29.964126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa4a2a39deb4e7ba54032acec14e029)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:29.964126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.014971 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa4a2a39deb4e7ba54032acec14e029'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.025866 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa4a2a39deb4e7ba54032acec14e029'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.026549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa4a2a39deb4e7ba54032acec14e029'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.027192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa4a2a39deb4e7ba54032acec14e029'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.041509 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-4728627d-363b-411a-a91d-ec306918e604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa4a2a39deb4e7ba54032acec14e029'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.055342 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1396/2790] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:45:29 2022] DELETE /identity/v3/users/daa4a2a39deb4e7ba54032acec14e029 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:30.236982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.249827 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fba20-3658-4ba2-b714-9cb9ff3f5b4e service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.314798 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.315675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.315955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.316230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] PATH_INFO: `/v3/projects/bc560e8b75194743bbb46d903b481357` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.317501 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc560e8b75194743bbb46d903b481357)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.319432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.369095 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc560e8b75194743bbb46d903b481357', 'request_id': 'req-302db88a-b7c2-4ba7-a069-e93fb49170e1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.381555 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc560e8b75194743bbb46d903b481357', 'request_id': 'req-302db88a-b7c2-4ba7-a069-e93fb49170e1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.388129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc560e8b75194743bbb46d903b481357', 'request_id': 'req-302db88a-b7c2-4ba7-a069-e93fb49170e1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.396027 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:30.400259 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] The token cache is being invalidate because project bc560e8b75194743bbb46d903b481357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:45:30.400491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.401265 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-302db88a-b7c2-4ba7-a069-e93fb49170e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:30.403105 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1395/2791] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:45:30 2022] DELETE /identity/v3/projects/bc560e8b75194743bbb46d903b481357 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:30.408664 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.408946 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.409221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.467741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:30.468719 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:30.512499 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:30.530272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1397/2792] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:30.539391 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.539686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.539969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.559219 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:30.560432 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:30.570208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1396/2793] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:30.574478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.584945 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8e020-59b0-46e0-bbb3-da0063a4ad26 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.590828 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.591592 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.591886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.592155 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.592961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80249) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:45:30.593268 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.594937 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.607546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1398/2794] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:45:30.612013 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.621466 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9845a-5052-43ed-b2ae-672c0ebb3b71 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.626773 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.627422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.627707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.627961 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.628649 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.630620 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.658720 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1397/2795] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:45:30 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:30.662905 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.674001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2213-e65a-429b-9167-a8c190f306ad admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.732194 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.732863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.733134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.733420 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.734123 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.736030 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.757330 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1399/2796] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:45:30 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:30.761815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.772910 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136ecd7-1d35-44b4-bb98-602bc90a0988 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.777588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.778244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.778498 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.778761 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.779491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.781181 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.791603 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1398/2797] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:30.796133 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.805722 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95ddae-2436-467f-8db5-51d4b00de3c0 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.810417 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e463ffb-0823-4cdf-ad62-0e1e937bf74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.811206 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463ffb-0823-4cdf-ad62-0e1e937bf74b admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.811754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463ffb-0823-4cdf-ad62-0e1e937bf74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.811754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e463ffb-0823-4cdf-ad62-0e1e937bf74b admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.812464 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e463ffb-0823-4cdf-ad62-0e1e937bf74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.814076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e463ffb-0823-4cdf-ad62-0e1e937bf74b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.823867 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1400/2798] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:30.828413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:30.838249 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c7a63-8c72-484b-850e-e66339dd2e02 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:30.843340 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:30.844223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.844487 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.844781 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] PATH_INFO: `/v3/projects/1b792decc6bc4f9eac472613b085a94c/users/5614fa9e38b8483096f183377b8e1a3e/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.860047 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b792decc6bc4f9eac472613b085a94c, user_id=5614fa9e38b8483096f183377b8e1a3e, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:30.862776 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:30.877281 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1399/2799] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:45:30 2022] PUT /identity/v3/projects/1b792decc6bc4f9eac472613b085a94c/users/5614fa9e38b8483096f183377b8e1a3e/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:30.887675 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884242f-36b7-48b7-935d-da10d407748f None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:30.888179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884242f-36b7-48b7-935d-da10d407748f None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:30.888292 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884242f-36b7-48b7-935d-da10d407748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:30.916056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-a884242f-36b7-48b7-935d-da10d407748f None None] MFA Rules not processed for user `5614fa9e38b8483096f183377b8e1a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:30.917605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-a884242f-36b7-48b7-935d-da10d407748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:30.985303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1401/2800] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:30.990116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.058163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb2181-d1a7-423d-afee-122298055194 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.064065 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.064732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.065016 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.065297 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.066046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.067985 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.097655 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1400/2801] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:45:30 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:31.102546 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884242f-36b7-48b7-935d-da10d407748f None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.114912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884242f-36b7-48b7-935d-da10d407748f None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.160552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:31.177733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.178338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.178617 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.178886 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.179580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.181517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.204075 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1402/2802] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:45:31 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:45:31.208637 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.219897 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4941ee0-5dd1-4f87-b48b-fc3631cfefb2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.224870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.225569 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.225853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.226132 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.226876 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.228495 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.239666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1401/2803] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:31.243980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.254807 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223cb30-39ea-4c9f-9af4-def6891919a2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.259813 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.260752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.261031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.261303 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] PATH_INFO: `/v3/projects/f6284aef271b4eeba30927c4096ac7d2/users/79b242a802bf49fcaf84b34efe969127/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.277715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6284aef271b4eeba30927c4096ac7d2, user_id=79b242a802bf49fcaf84b34efe969127, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.281247 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.298038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1403/2804] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:45:31 2022] PUT /identity/v3/projects/f6284aef271b4eeba30927c4096ac7d2/users/79b242a802bf49fcaf84b34efe969127/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:31.302172 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.313658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abb832-f2d5-4d5a-9bd8-22c10eacb10b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.376818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96e2238-c082-4f81-98e7-0be3885d9238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.377590 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2238-c082-4f81-98e7-0be3885d9238 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.377878 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2238-c082-4f81-98e7-0be3885d9238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.378160 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96e2238-c082-4f81-98e7-0be3885d9238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.378909 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e2238-c082-4f81-98e7-0be3885d9238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.380586 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96e2238-c082-4f81-98e7-0be3885d9238 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.391344 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1402/2805] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:31.395656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.407017 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd8e62-bffd-492b-b90b-58165ff0a26e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.412904 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.413834 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.414096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.414365 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] PATH_INFO: `/v3/projects/f6284aef271b4eeba30927c4096ac7d2/users/79b242a802bf49fcaf84b34efe969127/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.416557 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6284aef271b4eeba30927c4096ac7d2, user_id=79b242a802bf49fcaf84b34efe969127, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.419575 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.431126 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1404/2806] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:45:31 2022] PUT /identity/v3/projects/f6284aef271b4eeba30927c4096ac7d2/users/79b242a802bf49fcaf84b34efe969127/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:31.444244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.444627 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.444913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.457924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:31.472557 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] MFA Rules not processed for user `79b242a802bf49fcaf84b34efe969127`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:31.473809 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:31.549368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1403/2807] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:31.564728 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:31.566894 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:31.664522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d383915-7895-4f33-9319-5d7a75616314 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:31.669753 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d71c4-0d20-4d67-b8ef-ad514e4d998e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:31.670535 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d71c4-0d20-4d67-b8ef-ad514e4d998e service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:31.670821 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d71c4-0d20-4d67-b8ef-ad514e4d998e service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:31.671072 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d71c4-0d20-4d67-b8ef-ad514e4d998e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:31.681776 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d71c4-0d20-4d67-b8ef-ad514e4d998e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:31.683147 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d71c4-0d20-4d67-b8ef-ad514e4d998e service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:31.692797 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1405/2808] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:45:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:33.119963 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:33.181447 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657043c1-1a64-41c6-b751-c223f014f8bb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:33.187107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:33.187980 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:33.188257 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:33.188527 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] PATH_INFO: `/v3/users/5614fa9e38b8483096f183377b8e1a3e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:33.189934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5614fa9e38b8483096f183377b8e1a3e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:33.191901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:33.244792 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5614fa9e38b8483096f183377b8e1a3e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.257345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5614fa9e38b8483096f183377b8e1a3e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.258145 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5614fa9e38b8483096f183377b8e1a3e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.258755 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5614fa9e38b8483096f183377b8e1a3e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.265793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5614fa9e38b8483096f183377b8e1a3e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.280872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1404/2809] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:45:33 2022] DELETE /identity/v3/users/5614fa9e38b8483096f183377b8e1a3e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:33.285585 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:33.286048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:33.286162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:33.303907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:33.305085 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:33.366601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1406/2810] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:33.376660 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:33.378305 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:33.392201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:33.439267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1ab0d-5287-4941-acdd-e3ab1b5efc4b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:33.443789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:33.444429 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:33.444713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:33.444970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:33.455437 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:33.456890 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:33.468865 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1405/2811] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:33.751198 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:33.753059 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:33.765734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebbf9b-c25f-46d0-85ac-e5a25ab9cfa1 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:33.771297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:33.772181 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:33.772498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:33.772773 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] PATH_INFO: `/v3/projects/1b792decc6bc4f9eac472613b085a94c` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:33.774118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b792decc6bc4f9eac472613b085a94c)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:33.776043 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:33.793022 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:33.825892 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b792decc6bc4f9eac472613b085a94c', 'request_id': 'req-346de7f9-916a-444c-9caa-fbd365e2389a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.835160 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b792decc6bc4f9eac472613b085a94c', 'request_id': 'req-346de7f9-916a-444c-9caa-fbd365e2389a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.840961 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b792decc6bc4f9eac472613b085a94c', 'request_id': 'req-346de7f9-916a-444c-9caa-fbd365e2389a'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.857851 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] The token cache is being invalidate because project 1b792decc6bc4f9eac472613b085a94c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:45:33.858223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.859298 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:33.860920 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1407/2812] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:45:33 2022] DELETE /identity/v3/projects/1b792decc6bc4f9eac472613b085a94c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:33.865315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:33.875588 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2510ec-c25c-424f-ae12-1f27e4d4afaf service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:33.944686 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:33.945591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:33.945867 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:33.946129 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] PATH_INFO: `/v3/users/79b242a802bf49fcaf84b34efe969127` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:33.947444 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b242a802bf49fcaf84b34efe969127)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:33.949386 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.000148 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b242a802bf49fcaf84b34efe969127'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.011810 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b242a802bf49fcaf84b34efe969127'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.012401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b242a802bf49fcaf84b34efe969127'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.012992 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b242a802bf49fcaf84b34efe969127'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.019789 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-641b2aaa-f167-4b53-b267-9488569dd302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b242a802bf49fcaf84b34efe969127'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.035382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1406/2813] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:45:33 2022] DELETE /identity/v3/users/79b242a802bf49fcaf84b34efe969127 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:34.319459 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.320994 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:34.334941 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346de7f9-916a-444c-9caa-fbd365e2389a admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.396902 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.397804 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.398080 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.398352 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] PATH_INFO: `/v3/projects/f6284aef271b4eeba30927c4096ac7d2` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.399609 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6284aef271b4eeba30927c4096ac7d2)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.401618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.452739 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6284aef271b4eeba30927c4096ac7d2', 'request_id': 'req-b85e4ece-2836-413f-a4e0-f5752ad2b433'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.460174 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6284aef271b4eeba30927c4096ac7d2', 'request_id': 'req-b85e4ece-2836-413f-a4e0-f5752ad2b433'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.465917 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6284aef271b4eeba30927c4096ac7d2', 'request_id': 'req-b85e4ece-2836-413f-a4e0-f5752ad2b433'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.477751 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] The token cache is being invalidate because project f6284aef271b4eeba30927c4096ac7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:45:34.478041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.478811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-b85e4ece-2836-413f-a4e0-f5752ad2b433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:45:34.480533 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1408/2814] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:45:34 2022] DELETE /identity/v3/projects/f6284aef271b4eeba30927c4096ac7d2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:34.485758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.486280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.486280 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.505122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:34.506384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:34.539546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:34.574197 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1407/2815] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:45:34.582527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.582819 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.583096 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.604179 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:34.605445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:34.615895 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1409/2816] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:34.620516 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.631038 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b52755-caea-445e-9530-a858d2f42e39 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.637006 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.637717 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.638001 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.638273 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.639010 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:45:34.639319 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.641210 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.653024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1408/2817] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:45:34.657805 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.668090 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9ca1-780b-4320-a92a-8a1b96722dca None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.673560 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.674209 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.674488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.674760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.675450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.677390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.706188 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1410/2818] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:34.710247 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.720450 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01627554-3c95-421b-9a08-f1d02d36157c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.787413 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.788385 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.788730 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.789064 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.790115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.793368 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.819271 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1409/2819] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:34.823874 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.833196 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:34.837286 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7423b-92b1-499b-83a7-0bae63120a12 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.843798 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.844715 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.845041 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.845333 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.846238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.848243 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.860207 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1411/2820] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:34.864452 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.874785 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec64293-dcb3-490f-84da-dd452db6a8db admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.879837 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9ada8-d85f-4bd7-aa21-97168d41e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.880490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ada8-d85f-4bd7-aa21-97168d41e7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.880768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ada8-d85f-4bd7-aa21-97168d41e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.881042 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9ada8-d85f-4bd7-aa21-97168d41e7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.881817 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9ada8-d85f-4bd7-aa21-97168d41e7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.883490 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9ada8-d85f-4bd7-aa21-97168d41e7b7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.894104 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1410/2821] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:34.898430 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:34.908511 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2867464-905e-410c-ab0f-dfd0bec64b2d admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:34.913192 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:34.914113 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.914381 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.914644 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] PATH_INFO: `/v3/projects/2d5532d80a1a40cb8d51d5dec48d6215/users/1ebcf461488249b99f5b3fc7bfd2f0ae/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.930109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5532d80a1a40cb8d51d5dec48d6215, user_id=1ebcf461488249b99f5b3fc7bfd2f0ae, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:34.932741 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:34.948240 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1412/2822] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:45:34 2022] PUT /identity/v3/projects/2d5532d80a1a40cb8d51d5dec48d6215/users/1ebcf461488249b99f5b3fc7bfd2f0ae/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:34.959279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:34.959572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:34.959849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:34.988349 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] MFA Rules not processed for user `1ebcf461488249b99f5b3fc7bfd2f0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:34.989600 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:35.059861 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1411/2823] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:35.064643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.130445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69d28-7b95-4e14-8991-9008231abc53 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.135710 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.136366 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.136645 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.136915 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.137635 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.139624 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.166806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1413/2824] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:45:35 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:35.172793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.186969 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e520bd5-7dcf-4a99-b00f-d02ee059b3d8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.223032 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:35.258802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.259491 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.259775 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.260052 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.260749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.262760 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.284322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1412/2825] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:45:35 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:45:35.288527 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.300372 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d7e8-3e1d-4165-bdee-a9751beda86c admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.305980 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.306648 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.306930 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.307233 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.307966 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.309616 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.321355 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1414/2826] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:35.326901 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.337828 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af819c-fae6-4f61-967c-6f15b1c17c52 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.343629 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.344370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.344658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.344926 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] PATH_INFO: `/v3/projects/2ffa153d8df54762abb5d78fcb7ca959/users/e2994ce1f2dc41daa28c19613f406631/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.361127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa153d8df54762abb5d78fcb7ca959, user_id=e2994ce1f2dc41daa28c19613f406631, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.363978 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.378473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1413/2827] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:45:35 2022] PUT /identity/v3/projects/2ffa153d8df54762abb5d78fcb7ca959/users/e2994ce1f2dc41daa28c19613f406631/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:35.382481 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.392407 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d1bd0-bf4d-47f7-b3ab-700e55ae8648 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.448767 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fdb9b-6cd4-4f05-abd5-b34051251ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.449456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdb9b-6cd4-4f05-abd5-b34051251ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.449732 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdb9b-6cd4-4f05-abd5-b34051251ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.450040 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdb9b-6cd4-4f05-abd5-b34051251ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.450731 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fdb9b-6cd4-4f05-abd5-b34051251ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.452402 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fdb9b-6cd4-4f05-abd5-b34051251ca9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.462258 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1415/2828] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:45:35.467646 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.480080 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b5be7-b5e4-4e9f-ba51-3b39c14a6004 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.485758 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.486650 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.487220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.487220 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] PATH_INFO: `/v3/projects/2ffa153d8df54762abb5d78fcb7ca959/users/e2994ce1f2dc41daa28c19613f406631/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.489208 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa153d8df54762abb5d78fcb7ca959, user_id=e2994ce1f2dc41daa28c19613f406631, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.492478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.504069 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1414/2829] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:45:35 2022] PUT /identity/v3/projects/2ffa153d8df54762abb5d78fcb7ca959/users/e2994ce1f2dc41daa28c19613f406631/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:45:35.567863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.568277 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.568657 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.601217 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] MFA Rules not processed for user `e2994ce1f2dc41daa28c19613f406631`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:45:35.602494 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:35.635281 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:35.678078 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1416/2830] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:45:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:45:35.689509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:35.691640 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:45:35.756919 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37258bd-6506-496d-b2be-3861c5038d94 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:35.761115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:35.761793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:35.762081 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:35.762361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:35.823643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:35.825107 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:35.835178 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1415/2831] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:40.488780 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:45:40.499610 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ca319-f23a-4df4-a1b2-194ce93d8ca6 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:45:40.503818 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a897863-962b-4026-9b2f-e127405fc35b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:45:40.504453 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897863-962b-4026-9b2f-e127405fc35b service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:45:40.504730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897863-962b-4026-9b2f-e127405fc35b service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:45:40.505001 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897863-962b-4026-9b2f-e127405fc35b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:45:40.514427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a897863-962b-4026-9b2f-e127405fc35b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:45:40.515808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a897863-962b-4026-9b2f-e127405fc35b service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:45:40.525513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1417/2832] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:45:45.831004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:45:50.521672 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=22 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:01.509361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:01.582036 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96b852-cb43-4cd2-9965-69f5d3cb82df service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:01.587243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:01.588122 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:01.588401 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:01.588677 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] PATH_INFO: `/v3/users/1ebcf461488249b99f5b3fc7bfd2f0ae` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:01.590053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ebcf461488249b99f5b3fc7bfd2f0ae)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:01.592030 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:01.645366 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ebcf461488249b99f5b3fc7bfd2f0ae'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:01.657804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ebcf461488249b99f5b3fc7bfd2f0ae'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:01.658526 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ebcf461488249b99f5b3fc7bfd2f0ae'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:01.659161 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ebcf461488249b99f5b3fc7bfd2f0ae'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:01.666656 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ebcf461488249b99f5b3fc7bfd2f0ae'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:01.682057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1416/2833] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:46:01 2022] DELETE /identity/v3/users/1ebcf461488249b99f5b3fc7bfd2f0ae => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:01.686981 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:01.687496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:01.687496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:01.707579 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:01.708927 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:01.773532 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1418/2834] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:01.783533 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:01.785448 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:01.851571 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97380349-733c-4cda-aece-f3d2afefdfe0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:01.857358 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:01.858350 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:01.858691 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:01.859050 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:01.875102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:01.877282 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:01.888123 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1417/2835] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:02.174328 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:02.176087 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:02.189297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09969149-5a93-4c79-b753-fdfd594e6b01 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:02.195162 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:02.196038 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:02.196316 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:02.196589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] PATH_INFO: `/v3/projects/2d5532d80a1a40cb8d51d5dec48d6215` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:02.197832 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5532d80a1a40cb8d51d5dec48d6215)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:02.199734 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:02.248677 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5532d80a1a40cb8d51d5dec48d6215', 'request_id': 'req-5a014055-1541-47cb-8acf-d4c3f33f3807'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.257862 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5532d80a1a40cb8d51d5dec48d6215', 'request_id': 'req-5a014055-1541-47cb-8acf-d4c3f33f3807'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.264056 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5532d80a1a40cb8d51d5dec48d6215', 'request_id': 'req-5a014055-1541-47cb-8acf-d4c3f33f3807'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.275988 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] The token cache is being invalidate because project 2d5532d80a1a40cb8d51d5dec48d6215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:46:02.276272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.276914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.278496 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1419/2836] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:46:02 2022] DELETE /identity/v3/projects/2d5532d80a1a40cb8d51d5dec48d6215 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:02.282666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:02.293440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25e4a-196b-482f-a64f-6eaa7f902867 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:02.323201 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:02.366384 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:02.367174 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:02.367451 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:02.367724 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] PATH_INFO: `/v3/users/e2994ce1f2dc41daa28c19613f406631` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:02.368957 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2994ce1f2dc41daa28c19613f406631)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:02.370941 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:02.422643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2994ce1f2dc41daa28c19613f406631'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.435509 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2994ce1f2dc41daa28c19613f406631'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.436232 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2994ce1f2dc41daa28c19613f406631'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.436870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2994ce1f2dc41daa28c19613f406631'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.444262 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8bd5086-7984-47d0-8d71-9a9ee463242f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2994ce1f2dc41daa28c19613f406631'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.459950 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1418/2837] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:46:02 2022] DELETE /identity/v3/users/e2994ce1f2dc41daa28c19613f406631 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:02.758120 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:02.759717 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:02.774389 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a014055-1541-47cb-8acf-d4c3f33f3807 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:02.835152 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:02.836027 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:02.836315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:02.836615 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] PATH_INFO: `/v3/projects/2ffa153d8df54762abb5d78fcb7ca959` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:02.837910 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffa153d8df54762abb5d78fcb7ca959)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:02.839806 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:02.873118 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:02.890552 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffa153d8df54762abb5d78fcb7ca959', 'request_id': 'req-f19af39f-d740-44a9-9132-a4fd7ad042d1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.899730 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffa153d8df54762abb5d78fcb7ca959', 'request_id': 'req-f19af39f-d740-44a9-9132-a4fd7ad042d1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.906235 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffa153d8df54762abb5d78fcb7ca959', 'request_id': 'req-f19af39f-d740-44a9-9132-a4fd7ad042d1'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.918850 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] The token cache is being invalidate because project 2ffa153d8df54762abb5d78fcb7ca959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:46:02.919136 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.919830 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-f19af39f-d740-44a9-9132-a4fd7ad042d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:02.921597 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1420/2838] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:46:02 2022] DELETE /identity/v3/projects/2ffa153d8df54762abb5d78fcb7ca959 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:02.926975 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:02.927473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:02.927473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:02.947431 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:02.948635 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:03.012482 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1419/2839] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:03.020684 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.020975 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.021279 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.042205 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:03.043447 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:03.052747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1421/2840] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:03.057399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.068094 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2b897-7c6b-4607-bda5-b164beb2e03a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.073922 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.074593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.074872 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.075142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.075885 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:46:03.076188 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.077836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.090839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1420/2841] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:46:03.095478 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.105577 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50021266-faf2-48be-9eb7-6645e76f6d02 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.112446 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.113297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.113625 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.113898 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.114601 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.116514 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.146859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1422/2842] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:03.151090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.161778 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c96d44-d454-4845-89a9-0b97d8b4a611 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.189572 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:03.225090 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.225795 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.226083 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.226361 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.227060 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.229005 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.255584 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1421/2843] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:03.259549 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.270721 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd43de-c40c-4d88-b213-daa6fb5e1102 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.275474 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.276134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.276415 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.276691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.277435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.279048 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.289762 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1423/2844] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:46:03.294267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.304427 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecbfe2e-5412-4117-91a2-eb1638699c9a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.309224 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54e251-05a1-4a5c-963c-5fd4885849e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.310018 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54e251-05a1-4a5c-963c-5fd4885849e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.310267 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54e251-05a1-4a5c-963c-5fd4885849e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.310536 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54e251-05a1-4a5c-963c-5fd4885849e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.312538 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54e251-05a1-4a5c-963c-5fd4885849e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.312882 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54e251-05a1-4a5c-963c-5fd4885849e0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.323392 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1422/2845] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:46:03.327696 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.337922 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69848cff-00fd-4cc8-b5cc-43a2e9a1ef3e admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.343012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.343896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.344182 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.344454 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] PATH_INFO: `/v3/projects/d5fe2c46e60041f08ca3df70e711059a/users/d6c0bf2a983240d695cc6f3d012bc978/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.359586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fe2c46e60041f08ca3df70e711059a, user_id=d6c0bf2a983240d695cc6f3d012bc978, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.362307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.377426 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1424/2846] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:46:03 2022] PUT /identity/v3/projects/d5fe2c46e60041f08ca3df70e711059a/users/d6c0bf2a983240d695cc6f3d012bc978/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:46:03.394020 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.394399 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.394685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.423244 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] MFA Rules not processed for user `d6c0bf2a983240d695cc6f3d012bc978`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:03.424404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:03.495041 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1423/2847] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:03.499754 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.565779 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102ced9-a7ff-4564-8235-0ec6c870e6a2 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.572605 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.573295 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.573641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.573885 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.574588 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.576536 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.615849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1425/2848] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:03.620672 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.635029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1262dc9-c852-4a9e-a03f-e2aba76c7147 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.696946 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.697599 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.697889 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.698167 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.698873 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.700818 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.723934 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1424/2849] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/users => generated 306 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:03.730068 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.745238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc381b2-9264-4cae-bb39-7458f79e0f15 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.753134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.754110 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.754517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.754921 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.756004 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.757747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.772956 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1426/2850] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:46:03.779633 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.796102 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4450ae-9a0f-472c-8922-b6d366e7cd15 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.803238 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.804125 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.804408 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.804671 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] PATH_INFO: `/v3/projects/9a3b8b39304e4b378fbe271b0b9fbdfe/users/fea69c2d7c3f4099a476d32445454b45/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.827445 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3b8b39304e4b378fbe271b0b9fbdfe, user_id=fea69c2d7c3f4099a476d32445454b45, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.831612 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.847029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1425/2851] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:46:03 2022] PUT /identity/v3/projects/9a3b8b39304e4b378fbe271b0b9fbdfe/users/fea69c2d7c3f4099a476d32445454b45/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:03.851165 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.861849 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df5f8f-1a52-4e6d-896d-db42b06253b7 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.882323 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:03.922835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459644f-6426-4e0d-9268-0077dd4bdece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.923507 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459644f-6426-4e0d-9268-0077dd4bdece admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.923790 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459644f-6426-4e0d-9268-0077dd4bdece admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.924066 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459644f-6426-4e0d-9268-0077dd4bdece admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.924835 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459644f-6426-4e0d-9268-0077dd4bdece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.926473 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459644f-6426-4e0d-9268-0077dd4bdece admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.936580 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1427/2852] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:46:03.942070 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:03.947765 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:03.954622 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd669c-5f37-41bb-a5b2-072280c8d98b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:03.959784 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:03.960675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.960955 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.961246 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] PATH_INFO: `/v3/projects/9a3b8b39304e4b378fbe271b0b9fbdfe/users/fea69c2d7c3f4099a476d32445454b45/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:03.963230 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3b8b39304e4b378fbe271b0b9fbdfe, user_id=fea69c2d7c3f4099a476d32445454b45, role_id=0a3932a4f4b34d0d9a5d7bdbec41c8ca)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:03.965932 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:03.976664 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1426/2853] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:46:03 2022] PUT /identity/v3/projects/9a3b8b39304e4b378fbe271b0b9fbdfe/users/fea69c2d7c3f4099a476d32445454b45/roles/0a3932a4f4b34d0d9a5d7bdbec41c8ca => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:03.986990 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:03.987543 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:03.987876 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:04.023255 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] MFA Rules not processed for user `fea69c2d7c3f4099a476d32445454b45`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:04.024498 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:04.099661 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1428/2854] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:03 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:04.112850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:04.115136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:04.187033 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50e71a-78b7-4c77-a020-73915130ba4e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:04.191515 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:04.192190 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:04.192473 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:04.192748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:04.202618 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:04.204004 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:04.213951 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1427/2855] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:04.296792 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:04.307969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd84a-cda3-499c-badd-73b0366d4f80 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:04.311863 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd528ea2-abe2-4c20-af08-c282155b82c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:04.312506 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd528ea2-abe2-4c20-af08-c282155b82c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:04.312783 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd528ea2-abe2-4c20-af08-c282155b82c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:04.313070 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd528ea2-abe2-4c20-af08-c282155b82c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:04.369272 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd528ea2-abe2-4c20-af08-c282155b82c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:04.370764 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd528ea2-abe2-4c20-af08-c282155b82c6 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:04.380071 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1429/2856] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:08.490593 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:08.554023 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5dff6-ca1f-4e04-9559-7ee1993d58c9 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:08.559657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:08.560525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:08.560802 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:08.561034 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] PATH_INFO: `/v3/users/d6c0bf2a983240d695cc6f3d012bc978` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:08.563072 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c0bf2a983240d695cc6f3d012bc978)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:08.566338 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:08.627794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c0bf2a983240d695cc6f3d012bc978'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:08.640962 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c0bf2a983240d695cc6f3d012bc978'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:08.641675 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c0bf2a983240d695cc6f3d012bc978'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:08.642290 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c0bf2a983240d695cc6f3d012bc978'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:08.649982 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c0bf2a983240d695cc6f3d012bc978'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:08.668525 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1428/2857] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:46:08 2022] DELETE /identity/v3/users/d6c0bf2a983240d695cc6f3d012bc978 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:46:08.673513 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:08.674012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:08.674012 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:08.691599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:08.692810 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:08.735688 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=100 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:08.758311 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1430/2858] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:08.768223 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:08.769830 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:08.784436 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:08.833029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e3f9b-c137-4820-b8cd-de53542287f7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:08.837777 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:08.838422 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:08.838705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:08.838993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:08.848638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:08.850345 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:08.859993 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1429/2859] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:09.142014 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:09.143673 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:09.156429 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c3002-a6c8-4329-b244-f2b24a3294e4 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:09.161300 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:09.162219 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:09.162578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:09.162831 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] PATH_INFO: `/v3/projects/d5fe2c46e60041f08ca3df70e711059a` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:09.164135 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5fe2c46e60041f08ca3df70e711059a)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:09.166138 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:09.215842 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5fe2c46e60041f08ca3df70e711059a', 'request_id': 'req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.222833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5fe2c46e60041f08ca3df70e711059a', 'request_id': 'req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.228519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5fe2c46e60041f08ca3df70e711059a', 'request_id': 'req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.239824 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] The token cache is being invalidate because project d5fe2c46e60041f08ca3df70e711059a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:46:09.240119 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.240817 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.242589 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1431/2860] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:46:09 2022] DELETE /identity/v3/projects/d5fe2c46e60041f08ca3df70e711059a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:09.246666 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:09.256714 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394a13-4b97-490d-be75-579b03dac27b service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:09.317173 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:09.318082 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:09.318370 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:09.318702 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] PATH_INFO: `/v3/users/fea69c2d7c3f4099a476d32445454b45` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:09.319917 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea69c2d7c3f4099a476d32445454b45)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:09.321970 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:09.368685 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea69c2d7c3f4099a476d32445454b45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.380556 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea69c2d7c3f4099a476d32445454b45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.381286 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea69c2d7c3f4099a476d32445454b45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.381925 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea69c2d7c3f4099a476d32445454b45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.389274 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6adf152d-7527-4fdf-b9a5-fe3c46823a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea69c2d7c3f4099a476d32445454b45'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.404512 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1430/2861] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:46:09 2022] DELETE /identity/v3/users/fea69c2d7c3f4099a476d32445454b45 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:09.696177 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:09.697877 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:09.715641 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3f27e-f1d0-4ae5-b6a4-2ca5ffb7d459 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:09.782912 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:09.783794 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:09.784073 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:09.784338 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] PATH_INFO: `/v3/projects/9a3b8b39304e4b378fbe271b0b9fbdfe` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:09.785578 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3b8b39304e4b378fbe271b0b9fbdfe)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:09.787629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:09.845914 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3b8b39304e4b378fbe271b0b9fbdfe', 'request_id': 'req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.857370 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3b8b39304e4b378fbe271b0b9fbdfe', 'request_id': 'req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.866249 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3b8b39304e4b378fbe271b0b9fbdfe', 'request_id': 'req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.873307 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:09.878390 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] The token cache is being invalidate because project 9a3b8b39304e4b378fbe271b0b9fbdfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:46:09.878695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.879618 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-39b91f48-e37e-46ab-b9da-ddbdf6d306f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:09.881747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1432/2862] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:46:09 2022] DELETE /identity/v3/projects/9a3b8b39304e4b378fbe271b0b9fbdfe => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:09.887191 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621edf96-1179-45d4-99bb-d1068145944e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:09.887713 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621edf96-1179-45d4-99bb-d1068145944e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:09.887903 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621edf96-1179-45d4-99bb-d1068145944e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:09.907493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-621edf96-1179-45d4-99bb-d1068145944e None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:09.908706 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-621edf96-1179-45d4-99bb-d1068145944e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:09.941468 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:09.973377 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1431/2863] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:09.981811 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:09.982098 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:09.982369 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.001727 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:10.002999 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:10.012077 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1433/2864] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:10.016406 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621edf96-1179-45d4-99bb-d1068145944e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.026797 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621edf96-1179-45d4-99bb-d1068145944e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.032606 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.033263 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.033573 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.033846 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.034579 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:46:10.034887 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.036493 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.049382 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1432/2865] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:46:10.053968 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.063515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c3beb-99b3-49fa-8b32-cf9cb6eb9667 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.068479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.069170 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.069456 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.069875 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.070768 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.072724 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.106186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1434/2866] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:46:10 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:10.110371 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.121589 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9422b-fa3e-4621-85b6-4aab0d8f7d14 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.179643 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.180322 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.180602 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.180877 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.181591 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.183503 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.205896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1433/2867] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:46:10 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 13:46:10.209967 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.220725 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7dc99-2516-4284-ad7e-07811cd5c043 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.225156 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.225896 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.226231 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.226518 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.227260 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.228808 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.239697 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1435/2868] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:46:10.243793 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.253964 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf2d93-a144-486c-ae09-7d7cbd58a14a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.260136 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23b62c-1209-4760-b1d1-0034281d6f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.260843 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b62c-1209-4760-b1d1-0034281d6f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.261127 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b62c-1209-4760-b1d1-0034281d6f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.261441 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23b62c-1209-4760-b1d1-0034281d6f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.262608 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23b62c-1209-4760-b1d1-0034281d6f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.263804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac23b62c-1209-4760-b1d1-0034281d6f84 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.274853 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1434/2869] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:46:10.279704 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.293916 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c53c0-8ead-4b21-b9ed-dcd7abfb55b4 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.299712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.300586 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.300859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.301108 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] PATH_INFO: `/v3/projects/90a266ed0d98440189c9300dbffefac5/users/ad6ad05afb4d4465ab1c4661bf6e3610/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.316364 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a266ed0d98440189c9300dbffefac5, user_id=ad6ad05afb4d4465ab1c4661bf6e3610, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.319039 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.334743 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1436/2870] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:46:10 2022] PUT /identity/v3/projects/90a266ed0d98440189c9300dbffefac5/users/ad6ad05afb4d4465ab1c4661bf6e3610/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:10.347496 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.347815 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.348234 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.376936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] MFA Rules not processed for user `ad6ad05afb4d4465ab1c4661bf6e3610`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:10.378162 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:10.447115 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1435/2871] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:10.459452 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:10.461435 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:10.543955 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1297c-a2df-4702-bebf-a0b35b73af18 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:10.547733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb9ede-2db7-42a0-a79c-22e81be74bd6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ea7d2d4fd034e66aa589875e3e27ce8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:10.548378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9ede-2db7-42a0-a79c-22e81be74bd6 service glance] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:10.548662 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9ede-2db7-42a0-a79c-22e81be74bd6 service glance] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:10.548943 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb9ede-2db7-42a0-a79c-22e81be74bd6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:10.558238 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb9ede-2db7-42a0-a79c-22e81be74bd6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:10.559679 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb9ede-2db7-42a0-a79c-22e81be74bd6 service glance] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:10.569214 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1437/2872] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:20.443839 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:20.565131 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:36.560464 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:36.634705 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce8c3c-10ba-470d-9f70-70950e720ae3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:36.640147 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:36.641029 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.641312 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.641628 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] PATH_INFO: `/v3/users/ad6ad05afb4d4465ab1c4661bf6e3610` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.642913 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6ad05afb4d4465ab1c4661bf6e3610)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:36.644870 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:36.705740 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6ad05afb4d4465ab1c4661bf6e3610'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:36.720325 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6ad05afb4d4465ab1c4661bf6e3610'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:36.721053 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6ad05afb4d4465ab1c4661bf6e3610'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:36.721729 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6ad05afb4d4465ab1c4661bf6e3610'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:36.729022 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6ad05afb4d4465ab1c4661bf6e3610'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:36.743116 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1436/2873] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:46:36 2022] DELETE /identity/v3/users/ad6ad05afb4d4465ab1c4661bf6e3610 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:36.748013 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.748523 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.748640 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.767451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:36.768676 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:36.840929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1438/2874] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:36.851207 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:36.852862 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:36.914994 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c57ea4-501a-4e9d-95a4-763f88265e46 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:36.919836 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00b46d-e67c-45fe-aa47-4b3e5d32313b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:36.920484 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00b46d-e67c-45fe-aa47-4b3e5d32313b service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:36.920768 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00b46d-e67c-45fe-aa47-4b3e5d32313b service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:36.921057 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00b46d-e67c-45fe-aa47-4b3e5d32313b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:36.930850 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00b46d-e67c-45fe-aa47-4b3e5d32313b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:36.932315 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00b46d-e67c-45fe-aa47-4b3e5d32313b service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:36.941996 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1437/2875] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:37.233472 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.234969 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:37.247693 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d19eea-487a-4a96-8db6-0f412806e145 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.252712 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.253599 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.254023 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.254151 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] PATH_INFO: `/v3/projects/90a266ed0d98440189c9300dbffefac5` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.255337 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a266ed0d98440189c9300dbffefac5)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.260486 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.306669 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a266ed0d98440189c9300dbffefac5', 'request_id': 'req-88d6ad85-9acb-4775-948c-52506cc2190b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:37.316769 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a266ed0d98440189c9300dbffefac5', 'request_id': 'req-88d6ad85-9acb-4775-948c-52506cc2190b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:37.323063 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a266ed0d98440189c9300dbffefac5', 'request_id': 'req-88d6ad85-9acb-4775-948c-52506cc2190b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:37.334919 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] The token cache is being invalidate because project 90a266ed0d98440189c9300dbffefac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:46:37.335519 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:37.336344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-88d6ad85-9acb-4775-948c-52506cc2190b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:37.338031 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1439/2876] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:46:37 2022] DELETE /identity/v3/projects/90a266ed0d98440189c9300dbffefac5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:37.343086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.343373 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.343652 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.362707 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:37.363849 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:37.382924 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:37.429440 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1438/2877] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:46:37.437924 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.438237 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.438516 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.458650 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:37.459953 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:37.469993 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1440/2878] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:37.476546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.491663 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cc5c-70dc-48ed-b360-586477aeb133 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.497821 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.498475 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.498749 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.499024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.499748 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 13:46:37.500046 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.501709 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.514000 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1439/2879] 10.222.0.24 () {58 vars in 1152 bytes} [Thu May 19 13:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 13:46:37.518488 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.528747 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d4ed9-2ebf-45e2-a320-a7fe7bf9463b None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.534868 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.535517 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.535795 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.536062 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.536752 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.538691 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.567707 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1441/2880] 10.222.0.24 () {60 vars in 1152 bytes} [Thu May 19 13:46:37 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:37.572163 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.582400 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe8e82-a138-48e6-b115-c3e976a6c806 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.640061 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.640657 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.640954 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.641206 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.641930 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.643863 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.666469 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1440/2881] 10.222.0.24 () {60 vars in 1143 bytes} [Thu May 19 13:46:37 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 13:46:37.671145 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.681926 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023546-1df6-4112-b460-8f53014866cb admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.687525 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.688186 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.688463 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.688733 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.689480 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.691076 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.702378 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1442/2882] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 13:46:37.706615 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.717609 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66ac07-fd7b-4d0a-8c8e-2075fad6954b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.722570 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1590836-1cc3-443c-84bf-f66f0dc0d63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.723228 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1590836-1cc3-443c-84bf-f66f0dc0d63f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.723514 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1590836-1cc3-443c-84bf-f66f0dc0d63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.723794 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1590836-1cc3-443c-84bf-f66f0dc0d63f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.724535 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1590836-1cc3-443c-84bf-f66f0dc0d63f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.726561 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1590836-1cc3-443c-84bf-f66f0dc0d63f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.737113 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1441/2883] 10.222.0.24 () {58 vars in 1121 bytes} [Thu May 19 13:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 13:46:37.741942 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.752223 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0524e1-d5ee-445b-8e12-52d16bb9d959 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.757301 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.758221 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] REQUEST_METHOD: `PUT` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.758502 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.758778 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] PATH_INFO: `/v3/projects/a1a2c0f05bad42868061270430a80daf/users/968e855029e34acc8ad94f0009e15e60/roles/cc93bbcc688e4a08879b0b936b6018ea` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.773859 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a2c0f05bad42868061270430a80daf, user_id=968e855029e34acc8ad94f0009e15e60, role_id=cc93bbcc688e4a08879b0b936b6018ea)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:37.776515 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:37.792285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1443/2884] 10.222.0.24 () {60 vars in 1482 bytes} [Thu May 19 13:46:37 2022] PUT /identity/v3/projects/a1a2c0f05bad42868061270430a80daf/users/968e855029e34acc8ad94f0009e15e60/roles/cc93bbcc688e4a08879b0b936b6018ea => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 13:46:37.804086 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.804266 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.804404 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:37.835112 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] MFA Rules not processed for user `968e855029e34acc8ad94f0009e15e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:37.836658 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:37.907522 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1442/2885] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 13:46:37.919584 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:37.921451 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:37.944116 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:37.986656 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4a8de-0de8-4a78-b39e-e01b7412b973 admin admin] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:37.990643 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731b4a0-e7f2-4776-b942-d8916ca351e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd92108b779ca47aca8f941b7dfae80d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:37.991285 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731b4a0-e7f2-4776-b942-d8916ca351e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:37.991564 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731b4a0-e7f2-4776-b942-d8916ca351e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:37.991833 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731b4a0-e7f2-4776-b942-d8916ca351e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:38.000628 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731b4a0-e7f2-4776-b942-d8916ca351e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:38.002109 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4731b4a0-e7f2-4776-b942-d8916ca351e1 service cinder] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:38.011315 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1444/2886] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:45.126732 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:45.149243 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:45.196279 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab305240-0323-48f4-9578-a5b42bc4ab99 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:45.200915 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:45.201854 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:45.202139 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:45.202417 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] PATH_INFO: `/v3/users/968e855029e34acc8ad94f0009e15e60` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:45.203638 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968e855029e34acc8ad94f0009e15e60)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:45.205624 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:45.268936 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968e855029e34acc8ad94f0009e15e60'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.281134 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968e855029e34acc8ad94f0009e15e60'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.281845 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968e855029e34acc8ad94f0009e15e60'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.282449 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968e855029e34acc8ad94f0009e15e60'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.289337 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968e855029e34acc8ad94f0009e15e60'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.305184 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1443/2887] 10.222.0.24 () {58 vars in 1223 bytes} [Thu May 19 13:46:45 2022] DELETE /identity/v3/users/968e855029e34acc8ad94f0009e15e60 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:45.309844 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] REQUEST_METHOD: `POST` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:45.310344 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:45.310450 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:45.327803 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.auth.core [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] MFA Rules not processed for user `e50c121ca4544e98a148c3b6b1759f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80249) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 13:46:45.329050 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:45.392245 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1445/2888] 10.222.0.24 () {58 vars in 957 bytes} [Thu May 19 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 13:46:45.402683 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:45.405024 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:45.470920 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24dbcc-4ba0-4fe7-ba8f-8c55047b1595 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:45.474965 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84609c4a913144e2a71c508e96c53d0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:45.475611 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:45.475896 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:45.476175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:45.488858 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:45.491175 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:45.507316 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1444/2889] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:46:45.788522 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] Authenticating user token {{(pid=80249) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:46:45.790134 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.fernet_utils [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80249) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:46:45.803297 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33475-7215-4f52-8375-a5d14c5ff4e7 None None] Validating token access rules against request {{(pid=80249) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:46:45.808695 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e50c121ca4544e98a148c3b6b1759f0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '863bcb4c192047ed8ae210a081ac829f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80249) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:46:45.809646 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:46:45.809929 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:46:45.810445 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] PATH_INFO: `/v3/projects/a1a2c0f05bad42868061270430a80daf` {{(pid=80249) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:46:45.811774 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a2c0f05bad42868061270430a80daf)` {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:46:45.813907 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] RBAC: Authorization granted {{(pid=80249) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:46:45.866067 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a2c0f05bad42868061270430a80daf', 'request_id': 'req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.876479 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a2c0f05bad42868061270430a80daf', 'request_id': 'req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.882501 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a2c0f05bad42868061270430a80daf', 'request_id': 'req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b'} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.894760 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] The token cache is being invalidate because project a1a2c0f05bad42868061270430a80daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80249) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 13:46:45.895045 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.895846 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG keystone.notifications [None req-761cd318-08cb-4ee7-8e3e-eff3ea5a0e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80249) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 13:46:45.897629 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: [pid: 80249|app: 0|req: 1446/2890] 10.222.0.24 () {58 vars in 1232 bytes} [Thu May 19 13:46:45 2022] DELETE /identity/v3/projects/a1a2c0f05bad42868061270430a80daf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 13:46:55.500089 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:46:55.891427 dv-cinder-841125-6-iscsi devstack@keystone.service[80249]: DEBUG dbcounter [-] [80249] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80249) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 13:50:37.316456 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 13:50:37.318804 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:37.396149 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bfb94-df79-4721-a0de-f5274ef923d8 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 13:50:37.458546 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48ab1e052fff48bfbbc222528424b79c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9852768855f4cc39dc219f0c0ece5c0', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 13:50:37.459142 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 13:50:37.459478 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 13:50:37.459753 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 13:50:37.461892 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 13:50:37.474387 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 13:50:37.476003 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9049ca0-1ccb-4d81-a9c3-650d39dacc26 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 13:50:37.486504 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1445/2891] 10.222.0.24 () {60 vars in 1365 bytes} [Thu May 19 13:50:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 13:50:47.482045 dv-cinder-841125-6-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=58 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}